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INTRODUCTION 


Mobile  ground  targets  such  as  mobile  missile  launchers,  surface-to-air  threats,  tanks,  and  other 
vehicles  that  can  relocate  rapidly  represent  a  continuing  challenge  for  U.  S.  defense  forces.  Since  our 
involvement  in  Operation  Desert  Storm,  one  of  the  highest  U.  S.  defense  priorities  has  been  to  counter 
the  threat  posed  by  theater  missiles  (TMs).  Throughout  this  conflict,  detecting  and  locating  mobile 
missile  launchers  such  as  the  Scud-B  transporter-erector-launcher  (TEL)  proved  to  be  a  difficult  task  for 
air  defenses,  in  part  because  mobile  missile  launchers  are  able  to  move  to  a  safe  site  to  refuel  and  rearm 
within  three  to  five  minutes  of  a  launch.  Once  the  vehicle  has  left  the  launch  site,  it  may  be  impossible  to 
reacquire.  In  point  of  fact,  according  to  Fulghum  (1994),  not  a  single  Iraqi  TEL  was  destroyed  during  the 
Gulf  War.  Consequently,  the  U.vS.  Theater  Air  Defense  (TAD)  has  been  expanded  to  include  not  only 
aircraft  defense  but  also  defense  against  TMs  and  their  supporting  infrastructure. 

Various  sensor  technologies  (sensors,  sensor  management  subsystems,  automatic  target  cueing 
and  recognition  [ATC/ATR]  subsystems,  and  operator  interfaces)  are  expected  to  contribute  significantly 
to  attempts  to  improve  our  ability  to  find,  fix,  track,  target,  engage,  and  destroy  TMs  and  their  associated 
infrastructure.  Synthetic  aperture  radar  (SAR)  sensors,  on  both  surveillance  and  attack  platforms,  are  of 
particular  interest  since  this  type  of  sensor  is  able  to  function  at  all  times  of  day  and  night  and  in  adverse 
weather  conditions,  can  be  employed  over  long  standoff  ranges,  supports  accurate  geolocation  of 
detected/identified  targets,  and  is  capable  of  producing  the  level  of  image  quality  required  for  high 
confidence  target  acquisition  and  fratricide  avoidance.  Currently,  SAR  sensors  are  or  will  be  employed 
on  the  Joint  Surveillance  Target  Attack  Radar  System  (J-STARS  [E-8C])  developmental  surveillance 
system,  the  U-2S  reconnaissance  system,  several  developmental  unmanned  aerial  vehicle  (UAV) 
reconnaissance  systems,  and  the  F-15E,  B-IB,  and  B-2  attack  systems  to  support  navigation  and  weapon 
delivery.  The  use  of  SAR  imagery  for  target  detection,  acquisition,  and  destruction  can  be  affected  by  a 
number  of  factors,  including  image  resolution,  background  clutter  in  the  surrounding  terrain,  target 
orientation  and  the  presence  of  nontarget  objects  in  the  scene  that  may  be  easily  confused  with  the  target. 

Image  Resolution 

Image  resolution  refers  to  the  quality  of  the  imagery  produced  by  the  SAR  sensor.  At  very 
coarse  resolutions,  gross  features  such  as  size  and  shape  may  be  the  only  perceptible  characteristics  of  a 
potential  target.  At  higher  resolutions,  finer  details  of  the  object  can  be  discerned.  In  general,  as  image 
resolution  improves,  two  objects  can  occur  closer  in  proximity  to  each  other  and  still  be  perceived  as 
separate  objects  rather  than  as  a  single  entity.  Recent  investigations  using  both  actual  and  simulated 
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SAR  imagery  have  explored  the  impact  of  image  resolution  on  the  detection  of  mobile  ground  targets 
such  as  the  TEL,  revealing  that  performance  accuracy  typically  increases  as  the  resolution  improves 
(Davis,  See,  Shacklett,  &  Kuperman,  1996;  Kuperman,  Wilson,  &  Davis,  1993;  Kuperman,  Wilson,  & 
Perez,  1988;  See  &  Kuperman,  1995).  For  example,  using  actual  SAR  imagery  from  the  F-15E,  Davis  et 
al.  demonstrated  that  the  detection  rate  of  a  TEL  target  was  significantly  improved  at  a  resolution  of  4  ft 
by  6  ft  per  pixel  (M  =  94%)  as  compared  to  8.5  ft  by  8.5  ft  per  pixel  {M  =  87%). 

Other  studies  have  shown,  however,  that  such  performance  improvements  may  become 
negligible  as  resolution  increases  beyond  an  already  high  level  (Kuperman,  Wilson,  &  Davis,  1993; 
Kuperman,  Wilson,  &  Perez,  1988;  See  &  Kuperman,  1995).  In  the  See  and  Kuperman  investigation, 
resolutions  of  8.5  ft,  5  ft,  3  ft,  and  1  ft  were  examined  in  the  context  of  simulated  SAR  imagery.  The 
results  indicated  that  operators’  ability  to  detect  a  TEL  target  vehicle  improved  from  the  8.5  ft  resolution 
to  the  3  ft  resolution,  whereupon  no  further  improvements  in  performance  were  observed.  In  fact,  the 
only  indication  that  the  1  ft  resolution  might  represent  an  improvement  over  the  3  ft  resolution  was  the 
finding  that  operators  were  significantly  faster  at  correctly  rejecting  nontarget  images  at  the  finest 
resolution;  their  detection  ability  itself  did  not  differ  at  the  two  highest  resolutions  nor  did  their  reaction 
time  for  correct  detections  of  the  TEL  target. 

Background  Clutter 

Another  aspect  of  the  imagery  that  may  affect  detection  performance  is  background  clutter. 
Clutter  in  a  scene  is  often  defined  in  terms  of  the  “busyness”  of  the  background  in  which  a  potential 
target  is  embedded  and  may  include  both  natural  and  artificial  sources  (Toms  &  Kuperman,  1991).  For 
example,  high  clutter  may  be  characterized  by  the  presence  of  geographical  features  such  as  dense  forests 
that  can  make  a  target  more  difficult  to  discriminate  or  by  the  presence  of  large  numbers  of  artificial 
objects  (e.g.,  nontargets,  decoys,  and  other  target-like  objects).  Low  levels  of  clutter  might  be  presented 
by  a  desert  scene  with  minimal  vegetation  or  the  presence  of  few  confusing  man-made  objects. 

In  an  investigation  of  the  effects  of  background  clutter  on  the  detection  of  relocatable  targets  in 
simulated  SAR  imagery,  Kuperman,  Wilson,  and  Perez  (1988)  defined  clutter  as  the  amount  of  vegetative 
coverage  in  a  scene  and  examined  four  different  types:  (1)  a  forested  background,  representing  the 
greatest  overall  vegetative  coverage  and  tree  height;  (2)  tree  clutter,  with  moderate  coverage  and 
moderately  tall  trees;  (3)  bushes,  a  background  containing  short  trees  and  brush  with  only  a  few  tall  trees; 
and  (4)  a  river  background  containing  a  river  with  very  limited  forest  coverage.  Operator  performance 
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efficiency  was  lowest  in  the  forested  background  and  increased  progressively  as  the  background  changed 
from  trees  to  bushes  to  the  river;  i.e.,  as  the  amount  of  clutter  decreased. 

Several  other  studies  wherein  clutter  was  also  defined  in  terms  of  vegetative  coverage  have 
produced  similar  results.  For  example,  using  simulated  SAR  imagery,  See  and  Kuperman  (1995)  defined 
low,  medium,  and  high  levels  of  clutter  according  to  the  tree  density  in  the  scene.  Their  results  indicated 
that  the  hit  rate  for  detecting  a  TEL  was  not  affected  by  background  clutter;  however,  operators’  false 
alarms  did  increase  as  the  tree  density  increased  from  a  low  level  to  a  medium  or  high  level.  Thus,  as  the 
clutter  became  more  pronounced,  operators  were  more  inclined  to  incorrectly  designate  a  nontarget 
object  as  the  TEL  target.  Similarly,  in  two  other  studies  using  actual  SAR  imagery  collected  at  Eglin  Air 
Force  Base,  Florida,  performance  effectiveness  was  adversely  affected  by  background  clutter,  defined  in 
terms  of  both  the  amount  and  type  of  vegetative  coverage  (Davis,  See,  Shacklett,  &  Kuperman,  1996; 

See,  Davis,  &  Kuperman,  1997).  The  three  levels  of  clutter  included  in  both  studies  were  referred  to  as 
the  open  (low),  treeline  (medium),  and  sparse  (high)  background  sites.  The  results  in  both  studies 
indicated  that  the  hit  rates  for  detecting  and  for  locating  a  TEL  target  declined  as  the  clutter  increased; 
however,  the  false  alarms  did  not  increase  significantly. 

In  yet  another  study  of  background  clutter.  Cole  and  Hughes  (1984)  demonstrated  that  the 
presence  of  clutter  can  reduce  the  prominence  of  a  target,  interfering  with  both  its  attention-getting 
quality  (the  probability  that  the  object  will  be  noticed  when  the  observer  has  not  been  told  of  its  likely 
occurrence)  and  its  accessibility  to  search  (its  ability  to  be  quickly  and  reliably  located  when  the  observer 
is  specifically  directed  to  search  for  and  locate  it).  In  their  study,  Cole  and  Hughes  had  50  observers 
drive  a  vehicle  around  a  predetermined  route.  Thirty-five  targets,  disks  of  different  sizes  and 
reflectances,  were  positioned  along  the  route  where  normal  traffic  signs  might  be  located.  One  group  of 
observers  was  told  to  verbally  report  all  objects  that  attracted  their  attention  along  the  route  (attention 
conspicuity).  The  second  group  of  observers  was  told  to  verbally  report  the  presence  of  all  disk  targets 
and  conventional  road  traffic  devices  (search  conspicuity).  The  route  was  chosen  to  provide  a  variety  of 
normal  driving  environments  including  arterial  roads,  residential  streets,  and  shopping  centers  (the  most 
cluttered  of  the  three  environments).  The  results  indicated  that  differences  in  hit  rates  for  the  three  road 
types  were  highly  significant  for  both  attention  and  search  conspicuity  and  were  believed  to  be 
attributable  to  visual  clutter.  Specifically,  conspicuity  of  the  targets  was  reduced  in  the  shopping  center 
sections  of  the  route  as  compared  to  the  conspicuity  of  the  same  targets  in  the  arterial  road  and  residential 
sections.  Consequently,  both  attention  and  search  conspicuity  were  reduced  in  shopping  centers,  as 
revealed  by  lower  hit  rates. 
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Vehicle  Orientation 


Common  sense  dictates  that  target  vehicle  orientation  should  impact  the  operator’s  ability  to 
detect,  locate,  and  identify  a  potential  target.  Intuitively,  a  broadside  view  of  a  vehicle  should  yield  the 
greatest  level  of  performance  accuracy  since,  in  most  cases,  it  affords  the  best  view  of  the  object’s 
discriminating  features.  However,  the  effect  of  vehicle  orientation  with  ground  targets  has  not  yet  been 
studied  systematically;  hence,  the  impact  of  changes  in  orientation  remains  to  be  quantified.  To  date,  the 
only  relevant  studies  of  vehicle  orientation  and  detection/recognition  performance  have  been  conducted 
in  maritime  environments.  Whitehurst  (1976)  studied  the  effects  of  ship  orientation  on  the  range  at 
which  a  ship  could  be  recognized  as  a  merchant  ship  or  combat  ship.  The  targets  were  four  1 : 1250  scale 
models  of  merchant  and  combat  ships  videotaped  against  a  simulated  ocean  background  at  orientations  of 
10°,  30°,  50°,  70°,  and  90°  off  the  bow.  Simulated  distance  from  the  target  varied  from  36  km  to  8  km  in 
4  km  increments.  Participants  viewed  each  image  on  a  television  monitor  and  indicated  whether  the 
target  was  a  merchant  or  combat  ship.  The  results  revealed  that  the  probability  of  ship  recognition 
increased  as  the  orientation  of  the  ship  increased  from  10°  to  90°  (broadside)  across  all  ranges  examined. 
For  example,  at  a  range  of  26  km,  a  broadside  target  could  be  correctly  recognized  80%  of  the  time;  a 
target  oriented  at  10°,  only  about  10%  of  the  time.  Viewed  another  way,  the  results  showed  that  a  target 
could  be  correctly  recognized  80%  of  the  time  at  a  range  of  about  26  km  when  the  approach  was 
broadside.  By  comparison,  when  the  approach  was  10°  off  bow,  the  same  level  of  performance  could  be 
achieved  only  at  a  much  closer  range  of  8  km. 

In  a  similar  investigation,  Mocharnuk,  Gaudio,  and  Suwe  (1981)  examined  the  effects  of  ship 
orientation,  ship  type,  and  contrast  on  target  acquisition  with  simulated  infrared  imagery.  The  targets 
were  six  ships  positioned  either  bow  on  (head-on,  0°)  or  port  abeam  (broadside,  90°).  The  imagery  was 
constructed  by  photographing  1 : 1250  scale  models  of  the  ships  with  simulated  “white-hot”  polarity 
infrared  signatures  against  a  uniform  black  background.  The  ships  were  divided  into  two  classes 
according  to  their  country  of  origin.  During  the  task,  observers  viewed  a  dynamic  presentation  of  the 
imagery  at  starting  ranges  of  20,  22,  or  24  km,  with  a  constant  closing  rate  of  282  m/s.  At  the  end  of  each 
presentation,  observers  were  asked  to  categorize  the  ship  by  pressing  the  correct  key  on  a  response  box. 
Dependent  variables  included  range  at  classification  (simulated  range  to  the  target  when  the  recognition 
button  was  pressed)  and  accuracy  of  the  categorization.  With  respect  to  the  range  at  classification,  a 
multiple  regression  model  showed  that  ship  orientation  accounted  for  more  of  the  variance  (about  2 1  %) 
than  contrast,  ship  type,  and  accuracy.  Further  analysis  showed  that  observers  were  able  to  classify  the 
ships  at  greater  range  when  the  orientation  was  90°  as  opposed  to  0°.  Analyses  of  response  accuracy 
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produced  similar  results.  As  before,  ship  orientation  accounted  for  more  variance  than  did  ship  type. 
The  authors  hypothesized  that  the  effect  of  ship  orientation  was  probably  due  to  differences  in  both  the 
number  and  prominence  of  the  cues  that  are  present  at  the  different  target  orientations.  Following  the 
experiment,  observers  reported  that  relatively  few  good  cues  were  available  at  0°  (head-on)  as  compared 
to  90*^  (broadside).  At  0°,  the  cues  were  more  subtle  and  observers  had  to  rely  on  different  cues  such  as 
the  shape  and  width  of  the  bow,  making  classification  more  difficult. 

Confusers 


A  Definition  of  Confusers. 

An  operator’s  ability  to  detect  a  predesignated  target  can  depend  not  only  on  the  extent  to  which 
the  vehicle’s  features  are  discernible  given  its  orientation,  the  image  resolution,  and  the  background 
clutter  in  the  scene,  but  also  on  the  presence  of  other  objects  referred  to  as  “confusers.”  Greening  (1974) 
defines  confusers,  or  false  targets,  as  “objects  which  resemble  target  objects  to  a  greater  or  lesser  degree” 
(p.  25).  He  further  points  out  that  the  distinction  between  “clutter”  and  “confusers”  is  not  always  clear  in 
the  real  world.  The  distinction  can  be  made  only  in  relation  to  the  target  object  for  which  the  operator  is 
searching.  For  example,  pieces  of  clutter  that  actually  do  resemble  the  target  may  be  considered  confuser 
objects  that  will  require  fixation  and  evaluation  during  search  because  of  their  target-like  appearance. 
However,  if  targets  are  fairly  distinctive  (e.g.,  vehicles,  buildings,  bridges),  they  will  generally  not  be 
easily  confused  with  background  clutter.  In  this  case,  the  search  task  would  primarily  be  affected  by  the 
presence  of  other  vehicles  or  structures  similar  to  the  target  and  not  by  the  presence  or  absence  of  a  high 
level  of  texture  or  clutter  in  the  background.  In  essence,  the  principal  distinction  between  clutter  and 
confusers  is  whether  the  clutter  is  texture-like  (diffuse  irregular  background)  or  target-like.  The  former 
would  be  considered  clutter,  whereas  the  latter  would  be  considered  confusers.  Behaviorally,  the 
distinction  appears  to  be  between  objects  that  are  sufficiently  target-like  to  require  a  close  fixation  for  a 
classification  decision,  and  those  which  can  be  discarded  peripherally.  As  targets  become  less  isolated 
and  well-defined,  the  distinction  becomes  less  clear  cut. 

When  confusers  are  present  in  a  scene,  both  their  type  and  number  can  be  critical  to  operator 
performance.  The  type  of  confusers  is  important  because  decision-making  time  will  depend  on  the 
degree  of  similarity  of  each  confuser  to  the  anticipated  appearance  of  the  target  image.  A  confuser  that 
closely  resembles  the  target  may  require  several  fixations  before  a  decision  is  reached.  The  number  of 
confusers  is  also  important  because  decision  time  will  increase  as  the  number  of  nontarget  objects  to  be 
examined  and  rejected  increases  (Greening,  1974).  As  Greening  points  out,  a  number  of  early  studies 
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have  shown  that  the  time  to  acquire  a  target  increases  as  the  number  of  confusers  increases  and  as  the 
confusers  become  more  similar  to  the  target.  In  general,  the  average  search  time  for  a  target  among  a 
collection  of  nontargets  identical  to  one  another  and  differing  from  the  target  only  in  size  can  be 
determined  from  the  following  equation: 


t  oc 
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where  t  is  the  mean  response  time,  dg  is  the  angular  diameter  of  background  objects,  and  dr  is  the 
angular  diameter  of  targets.  Further,  the  number  and  type  of  confusers  may  affect  not  only  decision¬ 
making  time  but  also  detection  accuracy  itself.  After  inspecting  and  evaluating  the  objects  present  in  the 
scene,  the  operator  may  select  one  of  the  confusers  as  the  target  and  miss  the  actual  target  vehicle.  The 
corresponding  probability  of  detection  for  the  case  in  which  confusers  are  identical  to  one  another  and 
differ  from  the  target  only  in  size  can  be  obtained  by  means  of  the  following  equation: 


K  (d^-dj)- 
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where  Ps  is  the  single  glimpse  probability  of  detection;  A  is  the  area  to  be  searched;  m  is  an  empirically 
determined  constant;  ds  is  the  angular  diameter  of  background  objects;  and  dj  is  the  angular  diameter  of 
targets.  As  Greening  reports,  the  probability  of  fixating  a  nontarget  object  is  reduced  by  half  if  the  length 
differs  by  50%. 


A  Definition  of  Similarity. 

Given  that  confusers  are  defined  in  terms  of  their  similarity  to  a  predesignated  target,  it  is 
important  to  specify  how  the  degree  of  similarity  itself  is  determined.  O’ Kane,  Biederman,  Cooper,  and 
Nystrom  (1997)  provide  one  method  for  specifying  similarity  based  on  Biederman’s  (1987)  theory  of 
shape  representation,  which  holds  that  objects  can  be  represented  as  an  arrangement  of  viewpoint 
invariant  parts.  Viewpoint  invariant  shape  properties  are  features  of  edges  that  do  not  change  as  the 
object  is  rotated  in  depth;  consequently,  the  object’s  three-dimensional  shape  can  be  easily  ascertained 
from  its  two-dimensional  image  properties.  Examples  of  such  properties  include  whether  an  edge  is 
straight  or  curved,  whether  two  edges  are  parallel  or  not,  and  the  type  of  vertex  that  is  formed  when  two 
edges  meet.  Viewpoint-invariant  properties  can  be  contrasted  with  metric  properties,  which  do  vary  with 
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the  object’s  orientation.  Metric  properties  include  aspect  ratio  (length-to-width  ratio),  length,  and  degree 
of  curvature  of  an  edge.  Human  shape  recognition  is  largely  based  upon  the  recognition  of  viev^point- 
in variant  properties.  In  their  investigation.  O’ Kane  et  al.  (1997)  developed  a  similarity  metric  defined  by 
the  magnitude  and  number  of  viewpoint-invariant  part  differences  between  a  pair  of  vehicles  and 
demonstrated  its  ability  to  predict  confusion  rates  in  a  real-world  identification  task  involving  highly 
similar  objects. 

To  develop  the  similarity  metric,  a  “similarity  tree”  was  constructed  by  examining  near-range 
high-contrast  infrared  imagery  of  15  Russian  and  U.S.  military. vehicles.  For  each  vehicle,  salient,  easily 
detectable  features  that  could  distinguish  the  vehicles  from  one  another  were  noted:  wheels  or  tracks, 
presence  of  a  gun,  presence  of  a  cargo  bed,  etc.  The  primary  dimension  for  distinguishing  among  the 
vehicles,  which  would  serve  as  the  trunk  of  the  similarity  tree,  was  the  presence  of  wheels  versus  tracks. 
Other  features  of  the  wheeled  and  tracked  vehicles  provided  limbs  and  branches  on  the  similarity  tree 
that  could  further  distinguish  among  the  vehicles  within  each  category.  Using  the  similarity  tree,  the 
degree  of  similarity  between  any  two  vehicles  could  be  quantified  as  the  nodal  distance  between  them. 

For  example,  the  UAZ-469  and  the  M151  were  assigned  a  similarity  of  1  since  the  only  feature 
separating  them  was  the  presence  of  doors  on  the  cab.  On  the  other  hand,  the  UAZ-469  (wheeled)  and 
the  T62  (tracked),  which  were  separated  by  9  nodes  because  they  differed  on  a  number  of  dimensions, 
were  assigned  a  similarity  rating  of  9. 

Eighteen  male  tank  gunners  were  asked  to  view  each  thermal  image  and  identify  the  vehicle  by 
name  if  possible  or  by  basic-level  category  (tank,  armored  personnel  carrier,  and  truck)  if  the  exact 
vehicle  type  could  not  be  determined.  Of  those  trials  in  which  observers  provided  vehicle  identifications, 
55%  were  correct  responses  and  45%  were  confusion  errors.  Examination  of  the  confusion  errors  by 
nodal  distance  indicated  that  confusions  were  far  more  likely  to  occur  at  small  distances  (1,2,  and  3)  as 
opposed  to  large  distances  (7,  8,  and  9).  For  example,  of  the  total  number  of  confusions,  25%  occurred  at 
a  nodal  distance  of  1  and  fewer  than  5%  at  a  distance  of  9.  The  correlation  between  the  nine  nodal 
distances  and  the  mean  confusion  rates  at  each  distance  was  -.88.  That  is,  as  the  nodal  distance  between 
vehicles  decreased  (as  they  became  more  similar),  the  likelihood  of  their  confusion  increased. 

In  a  second  experiment  wherein  the  observers  were  forced  to  make  an  identification  response 
(i.e,,  were  not  permitted  to  use  basic-level  categories),  the  majority  of  confusion  errors  again  occurred  at 
the  three  smallest  nodal  distances.  At  the  three  largest  distances,  percentages  of  confusion  errors  were  all 
less  than  chance.  Additional  analysis  also  indicated  that  the  visibility  of  the  two  features  judged  to  be 
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most  critical  for  identifying  a  vehicle  correlated  at  .74  with  the  overall  percentage  of  correct  responses. 
When  factors  such  as  range,  size,  signal-to-noise  ratio,  and  orientation  were  added,  the  correlation 
increased  to  only  .79,  implying  that  the  more  critical  features  for  object  identification  were  the 
viewpoint-invariant  shape  properties  used  to  construct  the  similarity  tree.  In  sum,  the  results  of  the  two 
experiments  revealed  that  the  probability  of  confusing  one  object  with  another  could  be  predicted  from  a 
similarity  measure  based  on  the  number  of  the  discriminations  that  must  be  made  to  differentiate  the  two 
objects  as  well  as  their  prominence  or  visibility. 

Performance  Effects  of  Tar2et/Nontarget  (T/NT)  Similarity  and  Nontareet/Nontarget  (NT/NT) 
Similarity. 

In  terms  of  object  similarity  in  a  search  and  detection  task,  Duncan  and  Humphreys  (1989) 
demonstrated  that  the  similarity  of  nontarget  (confuser)  objects  to  one  another  (NT/NT  similarity)  is  as 
critical  to  performance  as  their  similarity  to  the  target  (T/NT  similarity).  They  reached  their  conclusion 
following  the  completion  of  a  series  of  experiments  in  which  alphabetic  characters  were  used  as  targets 
and  nontargets  (e.g.,  searching  for  a  45°-tilted  T among  nontarget  Ts  that  were  either  upright  or  rotated 
90°  clockwise).  Their  results  showed  first  that  search  difficulty,  as  indexed  by  search  time,  increased  as 
the  similarity  between  targets  and  nontarget  distractors  increased.  Thus,  it  became  harder  to  identify  the 
target  when  the  confusers  or  distractors  were  target-like  as  opposed  to  when  they  were  very  different 
from  the  target.  Second,  search  difficulty  increased  as  the  similarity  among  the  nontargets  themselves 
decreased.  That  is,  as  the  nontargets  became  more  varied,  it  took  longer  to  find  the  target  object.  Third, 
the  results  indicated  that  T/NT  and  NT/NT  similarity  interacted;  specifically,  increasing  T/NT  similarity 
had  little  effect  on  performance  when  NT/NT  similarity  was  high,  and  decreasing  NT/NT  similarity  had 
little  effect  when  T/NT  similarity  was  low.  Practically  speaking,  the  interaction  between  T/NT  and 
NT/NT  similarity  implies  that  performance  effectiveness  can  be  understood  only  by  considering  both 
variables  together.  The  best  scenario,  in  terms  of  rapid  search  time,  will  tend  to  occur  when  the  target 
and  nontargets  are  dissimilar  and  all  of  the  nontargets  are  similar  to  one  another.  In  this  situation,  the 
target  will  be  distinctive  because  it  will  differ  drastically  from  a  number  of  surrounding  objects  that  all 
look  the  same. 

In  an  attempt  to  explain  these  outcomes,  Duncan  and  Humphreys  developed  a  theory  based  on 
the  concepts  of  template  matching  and  spreading  suppression.  According  to  their  explanation,  visual 
input  from  a  scene  is  first  segmented  into  groups  of  items  that  are  similar  in  appearance  or  are  proximal 
to  one  another.  Thus,  items  in  a  scene  with  the  same  color,  motion,  or  shape  will  tend  to  group  together 
into  what  are  called  structural  units.  Structural  units  are  hierarchical.  Thus,  the  entire  scene  may  serve 
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as  one  structural  unit  that  is  subdivided  into  smaller  units  based  on  the  principles  of  similarity  and 
proximity.  These  in  turn  are  subdivided  into  smaller  structural  units  (e.g.,  individual  objects).  To  serve 
as  the  focus  for  further  action,  part  of  the  visual  input  must  be  selected  for  access  to  visual  short  term 
memory  (VSTM).  Duncan  and  Humphreys  propose  that  structural  units  act  as  wholes  to  compete  for  and 
gain  access  to  the  capacity-limited  VSTM.  Entry  to  VSTM  begins  with  a  template  matching  process 
used  to  classify  each  structural  unit  as  either  a  potential  target  or  nontarget.  That  is,  each  element  is 
matched  against  the  observer’s  template,  or  concept  of  the  target’s  appearance.  Those  units  that  most 
closely  match  the  template  receive  the  highest  weights  and  are  most  likely  to  enter  VSTM  and  become 
the  focus  of  current  behavior  for  further  processing.  This  aspect  of  the  theory  explains  why  increasing 
T/NT  similarity  is  detrimental  to  visual  search.  As  nontargets  become  more  similar  to  the  target  object, 
they  more  closely  match  the  observer’s  target  template  and  therefore  receive  high  weights  that  may 
parallel  the  weight  given  to  the  actual  target.  Consequently,  nontargets  and  targets  will  compete  equally 
for  access  to  VSTM;  ultimately,  the  target  may  not  be  the  focus  of  current  behavior. 

The  concept  of  spreading  suppression  is  used  to  explain  why  high  NT/NT  similarity  is  beneficial. 
According  to  the  theory,  structural  units  possess  what  is  referred  to  as  weight  linkage.  Thus,  a  change  in 
weight  for  one  unit  will  be  distributed  to  other  units  in  proportion  to  the  strength  of  their  grouping. 
Consequently,  such  units  tend  to  gain  or4ose  entry  to  VSTM  together.  Weight  linkage  translates  into 
efficient  rejection  of  strongly  grouped  nontargets  through  a  process  of  spreading  suppression.  Hence, 
when  the  nontargets  are  highly  similar  to  one  another,  they  can  be  rejected  as  a  group,  speeding  the 
search  process.  When  nontargets  are  dissimilar,  they  will  not  be  strongly  grouped,  their  weights  will 
vary  depending  on  their  degree  of  similarity  to  the  target,  and  they  will  have  to  be  examined  and  assessed 
independently.  Decreasing  NT/NT  similarity  reduces  the  opportunity  for  spreading  suppression  of 
grouped  nontargets. 

In  summary,  according  to  the  theory,  there  are  two  sufficient  conditions  for  efficient  visual 
search.  The  first  is  that  NT/NT  similarity  should  be  greater  than  T/NT  similarity.  If  this  is  so,  the 
nontargets  will  form  strong  groups  with  low  weights;  spreading  suppression  will  enable  them  to  be 
rejected  for  access  to  VSTM,  leaving  only  the  target  object.  The  second  condition  is  that  T/NT  similarity 
should  be  low  so  that  nontarget  weights  will  be  negligible;  again  the  target  will  have  the  greatest 
likelihood  of  entering  VSTM.  As  the  authors  note,  however,  their  ideas  were  developed  on  the  basis  of 
highly  artificial  ’laboratory  search  tasks;  hence,  it  remains  to  be  seen  whether  these  notions  generalize  to 
the  real  world. 
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One  indication  that  they  might  apply  to  real-world  tasks  comes  from  a  study  conducted  by 
Birkmire,  Karsh,  Barnette,  Pillalamarri,  and  Breintenbach  (1991).  The  authors  examined  the 
confusability  of  the  target  signatures  of  four  tracked  and  four  wheeled  military  vehicles  embedded  in  one 
of  twelve  different  levels  of  white  noise  designed  to  simulate  actual  sensor  imagery.  Detection 
(perception  of  a  possible  target  vehicle),  recognition  (perception  of  target  category — wheeled  or  tracked), 
and  identification  (determination  of  actual  target  name — e.g.,  Abrams  tank)  performance  were  assessed. 
On  average,  operators  took  only  4.5  seconds  to  detect  a  target  and  verbalize  its  identity  and  location  in  an 
image.  With  respect  to  performance  accuracy,  detection  and  recognition  performance  were  similar  for 
the  wheeled  and  tracked  vehicles;  however,  the  wheeled  vehicles  were  more  easily  identified  than  the 
tracked.  The  authors  postulated  that  the  greater  degrees  of  similarity  among  the  tracked  targets  made 
them  more  easily  confusable  with  one  another  than  was  true  for  the  wheeled  targets. 

The  authors  also  point  out  that  confusion  between  two  targets  in  a  set  depends  not  only  on  the 
amount  of  features  they  jointly  share  but  also  on  how  those  features  are  shared  with  other  targets  in  the 
set.  For  example,  in  their  study,  the  Abrams  tank  target  elicited  many  more  “Abrams”  responses  than 
“Concept”  responses;  however,  when  the  Concept  tank  appeared,  both  “Abrams”  and  “Concept” 
respon.ses  occurred  equally  often.  This  performance  difference  stemmed  from  the  differences  in 
similarity  of  each  target  to  the  other  two  tanks  (Bradley  and  Test  Bed)  in  the  set.  Namely,  the  Abrams 
tank  had  some  features  in  common  with  all  three  of  the  other  tanks  and  elicited  an  equal  number  of 
incorrect  “Bradley,”  “Concept,”  and  “Test  Bed”  identifications  when  it  was  presented;  these  false 
identifications,  however,  were  still  significantly  less  than  the  number  of  correct  responses  to  the  Abrams 
tank.  The  Concept  tank,  on  the  other  hand,  shared  features  primarily  with  only  two  of  the  other  tanks  in 
the  set  (Bradley  and  Abrams),  and  these  two  tanks  were  each  named  just  as  often  as  the  Concept  when 
the  Concept  tank  appeared;  hence,  the  number  of  correct  responses  did  not  exceed  the  false 
identifications.  Such  outcomes  imply  that  if  the  target  set  is  changed,  the  relative  criticality  of  a  given 
feature  will  change  and  will  be  reflected  in  the  overall  pattern  of  the  confusion  matrix.  In  other  words,  to 
use  Duncan  and  Humphreys’  (1989)  phraseology,  “confusability”  will  depend  on  both  T/NT  similarity 
and  NT/NT  similarity. 

Recent  Studies  of  Confusers  in  Actual  and  Simulated  Environments. 

Several  recent  studies  examining  the  effects  of  number  of  confusers  on  operator  performance 

have  corroborated  the  general  conclusion  that  detection  time  and  accuracy  deteriorate  as  the  number  of 
confusers  increases  (Greening,  1974),  demonstrating  its  applicability  in  various  types  of  realistic  and 
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artificial  environments.  For  example,  Boersema  and  Zwaga  (1985)  examined  the  effects  of  number  of 
advertisements  on  the  search  for  a  target  routing  sign  in  a  railway  station.  The  target  routing  sign  was 
white-on-blue.  The  distractor  advertisements  varied  in  number  and  size,  but  none  was  blue.  The  results 
showed  that  the  presence  of  the  advertisements  was  detrimental  to  search  performance;  however,  the 
introduction  of  only  one  advertisement  in  a  scene  tended  to  reduce  the  percentage  of  correct  detections  of 
the  routing  sign  more  so  than  the  additional  introduction  of  a  second  and  third  advertisement. 

In  a  subsequent  study,  Boersema,  Zwaga,  and  Adams  (1989)  again  examined  the  effects  of 
distractor  advertisements  on  target  search  for  routing  signs  using  color  slides  of  scenes  in  railway  stations 
and  other  public  environments  for  their  stimuli.  Each  scene  contained  routing  signs  with  white  lettering' 
on  a  blue  rectangle  as  well  as  zero,  one,  or  three  advertisements.  The  ads  were  all  rectangular,  but  none 
was  blue  in  color.  Upon  viewing  each  slide,  participants  were  instructed  to  find  the  target  destination  as 
quickly  as  possible  and  its  associated  arrow  in  the  scene’s  routing  information  before  pressing  a  button  to 
terminate  the  scene.  Subsequently,  they  verbally  reported  the  direction  of  the  arrow.  In  two  of  the  three 
scenes  examined,  search  time  (finding  and  fixating  on  the  sign)  increased  systematically  with  the  number 
of  advertisements;  however,  processing  time  (reading  the  sign  and  deciding  on  the  response)  and  overall 
reaction  time  were  unaffected.  Analysis  of  the  eye  movement  data  showed  that  the  number  of  eye 
fixations  in  those  two  scenes  also  increased  systematically  with  the  number  of  distractors.  Thus,  the 
results  indicated  that  the  search  time  for  relevant  information  increased  as  the  number  of  distractors 
increased  due  to  an  increase  in  the  number  of  fixations  during  search.  However,  further  analysis 
indicated  that  only  some  of  the  additional  fixations  actually  fell  on  the  advertisements.  Hence,  the 
increased  number  of  fixations  was  not  simply  due  to  the  presence  of  more  distractor  objects.  The  authors 
postulated  that  the  introduction  of  distractors  in  the  scene  results  in  less  efficient  scanning. 

Consequently,  the  increase  in  search  time  when  ads  are  introduced  only  partly  consists  of  time  spent 
actually  fixating  the  distractors. 

In  a  third  investigation,  Boersema  and  Zwaga  (1993)  studied  the  effect  on  search  performance  of 
the  number  of  distractors  in  an  artificial  environment.  Images  consisted  of  a  target  object  and  zero,  one, 
or  three  distractors  superimposed  on  a  continuous  background  of  gray  and  pastel  triangles.  The  target 
and  distractors  consisted  of  rectangular  areas  with  one  or  more  strings  of  simple  symbols  inserted  in 
them.  The  target  area  was  always  blue  with  white  symbols  and  always  contained  a  small  white  arrow 
pointing  left,  right,  up,  or  down.  Participants  were  required  to  examine  each  image  and  press  a  button  as 
soon  as  they  located  the  target;  subsequently,  they  verbally  reported  the  direction  of  the  arrow  in  the 
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target.  As  in  the  previous  study  with  realistic  scenes,  both  search  time  and  the  number  of  eye  fixations 
during  search  increased  systematically  with  the  number  of  distractors  in  two  of  the  three  scenes  used. 

Holahan,  Culler,  and  Wilcox  ( 1 978)  conducted  a  similar  type  of  investigation  in  which  they 
examined  the  effects  of  number  of  distractors  in  a  simulated  traffic  environment.  The  target  was  an 
octagonal  replica  of  a  standard  red  stop  sign  with  white  lettering.  The  distractors  were  square  replicas  of 
commercial  signs  with  white  lettering  on  either  a  red,  orange,  blue,  green,  or  black  background.  A 
different  four-letter  word  was  printed  on  each  distractor  sign.  For  each  35  cm  x  25  cm  scene,  the  number 
of  distractors  was  either  2,  4,  6,  or  10;  they  were  located  either  proximate  to  the  target  (no  further  than 
1 1 .4  cm  from  the  target)  or  distant  from  the  target  (no  closer  than  1 1 .4  cm  to  the  target).  The  target  was 
either  present  (requiring  a  “stop”  response  from  the  observer)  or  absent  (requiring  a  “go”  response  from 
the  observer).  The  results  indicated  that  observers’  mean  reaction  time  increased  systematically  as  the 
number  of  distractors  in  the  scene  increased.  Further,  reaction  time  was  significantly  elevated  when 
distractors  were  proximate  to  the  target,  indicating  that  the  location  of  distractors  relative  to  the  target  is 
critical  to  a  driver’s  ability  to  discriminate  a  traffic  sign  effectively. 

Finally,  Pashler  (1987)  studied  the  effects  of  the  number  of  target-confusable  distractors  on 
visual  search  performance  with  alphabetic  characters.  Target-distractor  similarity  was  manipulated 
between  trials  so  that  observers  could  not  anticipate  the  discriminability  level  on  a  given  trial.  Observers 
searched  for  a  target  letter  among  a  circular  display  of  six  items  and  indicated  whether  it  was  present  or 
not  on  each  trial.  The  primary  variables  of  interest  were  target  presence/absence  and  the  number  of 
target-confusable  distractors  (0,  1,  3,  or  5).  The  target  was  the  capital  letter  C;  similar  distractors  were 
capital  Gs  and  dissimilar  distractors  were  capital  Xs  and  Ls.  The  results  indicated  that  as  the  number  of 
target-confusable  distractors  increased,  both  the  reaction  time  for  correct  responses  and  the  error  rate 
(misses  and  false  alarms)  increased.  In  addition,  reaction  time  increased  more  sharply  on  target-absent 
trials.  The  authors  explained  these  results  by  means  of  a  two-stage  model  of  visual  search.  Initial 
processing  takes  the  form  of  a  parallel  analysis  of  the  display  during  which  each  display  position  is 
analyzed  for  its  similarity  to  the  target.  If  target-distractor  similarity  is  low  or  if  the  target  and 
background  are  well-learned,  the  observer  might  respond  on  the  basis  of  the  first  stage.  Otherwise, 
processing  proceeds  to  the  second  stage,  which  involves  a  sequential  examination  of  each  item  in  the 
display  exceeding  a  certain  value  of  “target-likeness.”  The  serial  examination  continues  until  a  target  is 
located,  or  until  each  item  has  been  examined  and  the  target  is  determined  to  be  absent.  Thus,  according 
to  the  model,  as  the  number  of  target-like  objects  increases,  reaction  time  increases  because  this  second 
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stage  of  processing  is  required  for  more  items.  Further,  reaction  time  increases  more  on  target-absent 
trials  because  every  item  must  be  examined  sequentially  before  a  decision  can  be  made. 

The  Theory  of  Signal  Detection 


Sensitivity  and  Bias. 

Reaction  time  and  the  percentage  of  correct  detections  are  the  most  common  dependent  variables 
in  many  studies  of  target  detection  performance.  A  somewhat  more  sophisticated  method  for  assessing 
performance  effectiveness  in  many  types  of  detection  tasks,  including  target  acquisition,  comes  from  a 
model  of  perceptual  processing  known  as  the  theory  of  signal  detection  (TSD)  (Gescheider,  1985;  Green 
&  Swets,  1966;  Macmillan  &  Creelman,  1991;  Wilson,  1992).  A  TSD  application  entails  the  derivation 
of  two  independent  measures  of  performance:  perceptual  sensitivity  and  response  bias.  The  d'  index 
of  sensitivity  is  a  perceptual  measure  that  provides  a  bias-free  estimate  of  the  operator’s  ability  to 
discriminate  targets  from  nontargets.  The  index  of  response  bias,  c,  provides  an  independent  assessment 
of  the  operator’s  general  willingness  to  make  a  detection  (“target”)  response,  which  can  vary  on  a 
continuum  from  conservative  to  lenient.  Both  measures  are  derived  from  operators’  hits  (correct 
detections)  and  false  alarms  (errors  of  commission)  during  the  course  of  a  task.  A  detection  theory 
analysis  is  preferable  to  separate  examinations  of  hits  and  false  alarms  since  it  permits  performance  to  be 
characterized  independently  in  terms  of  sensing  abilities  and  decision  making  processes  with  measures 
that  simultaneously  take  both  the  hits  and  false  alarms  into  account,  as  reflected  in  the  computing 
formulae  for  sensitivity  and  bias: 


^  =  [3] 

c  =  .5  (zfa  +  ^h)  [4] 

In  each  formula,  z  represents  the  standard  normal  deviate  associated  with  proportions  of  hits  (H)  and 
false  alarms  (FA),  both  of  which  enter  directly  into  the  derivation  of  each  TSD  index. 

The  derivation  of  each  index  is  portrayed  graphically  in  Figure  1 .  According  to  TSD,  the 
detection  of  a  target  is  said  to  occur  against  a  background  of  noise  that  is  always  present.  The  levels  of 
sensory  activation  produced  by  the  noise  (N,  nontargets)  and  the  signal-plus-noise  (SN,  targets)  are 
assumed  to  distribute  normally  with  unit  variance.  The  operator's  ability  to  differentiate  between  the 


13 


signal  and  noise,  cT,  is  represented  by  the  magnitude  of  the  distance  between  the  two  distributions.  As  the 
operator's  ability  to  differentiate  the  target  from  nontargets  increases,  the  distance  between  the  noise  and 
signal-plus-noise  distributions  increases,  and  d  will  increase.  In  general,  ct  scores  range  from  0  (no 
ability  to  discriminate  targets  from  nontargets)  to  about  4  (near  perfect  performance).  The  operator's 
willingness  to  make  a  detection  response,  c,  is  the  distance  of  the  criterion  from  the  intersection  of  the 
two  distributions  in  standard  units.  The  criterion  represents  the  operator's  cutoff  for  responding  “target” 
(activation  levels  that  exceed  the  criterion)  versus  “nontarget.”  Hence,  when  the  criterion  is  located  at 
the  intersection,  bias  will  be  neutral  (c  =  0)  since  “target”  and  “nontarget”  responses  will  be  equally 
probable.  Bias  will  be  conservative  (positive  c  scores)  if  the  criterion  is  located  to  the  right  of  the 
intersection  and  lenient  (negative  c  scores)  if  it  is  to  the  left. 


Figure  1:  The  TSD  indices  of  sensitivity  {d')  and  response  bias  (c). 

In  many  detection  tasks  in  which  TSD  is  applied,  including  target  acquisition,  observers  may  be 
required  not  only  to  detect  the  presence  of  a  target  but  also  to  designate  its  location.  Thus,  once  they 
have  determined  that  a  target  is  present,  observers  must  decide  which  of  several  alternative  “target-like” 
objects  present  in  the  scene  has  the  greatest  probability  of  being  the  target.  The  probability  of  correctly 
designating  the  target’s  location  when  it  is  present  is  derived  from  the  Joint  probability  of  making  both  a 
correct  detection  of  the  target  and  a  correct  identification  of  its  location.  Under  these  circumstances,  it  is 
still  possible  to  apply  TSD  and  obtain  estimates  of  operator  sensitivity  and  bias,  with  some  modification. 
The  cf  index  of  operator  sensitivity  can  be  interpreted  as  the  operator's  ability  to  differentiate  the  actual 
target  from  other  alternative  “target-like”  objects  that  may  be  present.  It  is  estimated,  from  either  a 
computational  formula  or  tables  of  d,  on  the  basis  of  the  number  of  alternatives  available  for  designation 
and  the  operator’s  ensuing  proportion  of  correct  localizations  (Hacker  &  Ratcliff,  1979;  Macmillan  & 
Creelman,  1991).  The  index  of  bias  in  a  target  localization  task  provides  a  measure  of  the  operator’s 
degree  of  caution  or  conservatism  in  making  the  designation  response.  Its  calculation  is  the  same  as  that 
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for  target  detection,  with  the  proportion  of  correct  localizations  substituted  for  hits  (Macmillan  & 
Creelman,  1991). 

Receiver  Operatin2  Characteristic  Curves. 

In  addition  to  the  calculation  of  d  and  c,  the  performance  results  of  a  target  acquisition  task  are 
commonly  portrayed  through  the  receiver  operating  characteristic  (ROC)  curve  (Gescheider,  1985;  Green 
&  Swets,  1966;  Macmillan  &  Creelman,  1991).  An  ROC  curve  for  target  detection  represents  the 
tradeoffs  in  hits  and  false  alarms  for  a  given  level  of  sensitivity  as  the  response  bias  shifts  from 
conservative  to  lenient.  Examples  of  ROC  curves  for  target  detection  are  depicted  in  Figure  2.  The 
diagonal  line  in  the  figure  represents  a  case  in  which  the  operator  is  unable  to  discriminate  targets  from 
nontargets  {d  =  0).  For  higher  levels  of  sensitivity,  the  ROC  curve  is  displaced  further  from  the  chance 
diagonal,  as  can  be  seen  in  the  curves  in  Figure  2  for  d  values  of  .5,  1 ,  2,  and  3.  Thus,  movement 
between  curves  represents  changes  in  operator  sensitivity,  or  detection  capability.  Movement  in  a  left-to- 
right  direction  along  a  single  curve  represents  a  change  in  criterion  from  conservative  to  lenient  for  that 
level  of  sensitivity.  In  a  practical  setting,  ROC  curves  are  useful  for  determining  tradeoffs  between  hits 
and  false  alarms  both  within  and  between  given  levels  of  sensitivity. 


Figure  2:  Receiver  operating  characteristic  curves  showing  d'  values  from  0  to  3. 

In  the  case  of  a  target  acquisition  task  that  involves  both  detection  and  localization,  more  than 
one  type  of  ROC  curve  must  be  generated  to  provide  a  comprehensive  assessment  of  observers’ 
performance.  First,  the  conventional  ROC  curve  plotting  the  proportion  of  hits  as  a  function  of  the 
proportion  of  false  alarms  is  obtained.  In  addition,  a  separate  curve  for  localization  performance, 
referred  to  as  the  localization  receiver  operating  characteristic  (LROC),  is  used  to  portray  the  proportion 
of  correct  localizations  as  a  function  of  the  proportion  of  false  alarms  (Macmillan  &  Creelman,  1991; 
Starr,  Metz,  Lusted,  &  Goodenough,  1975;  Swensson  &  Judy,  1981 ;  Swets  et  al.,  1979). 
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Examples  of  ROC  curves  for  a  detection-plus-localization  task  are  portrayed  in  Figure  3.  The 
ROC  curve  for  detection  is  derived  in  the  usual  manner  from  proportions  of  hits  and  false  alarms.  The 
LROC  curve  for  localization,  on  the  other  hand,  is  obtained  from  only  those  hits  that  have  been  correctly 
localized.  Thus,  the  LROC  depicts  the  functional  relation  between  proportions  of  correctly  localized  hits 
and  proportions  of  false  alarms  as  the  criterion  for  a  positive  detection  decision  varies  from  conservative 
to  lenient.  As  exemplified  by  the  curves  in  Figure  3,  LROC  curves  will  fall  short  of  the  concomitant 
ROC  curve  for  detection,  due  to  the  fact  that  some  of  the  correct  “target”  responses  on  target  trials  will 
not  be  accompanied  by  correct  localizations.  Such  responses  would  be  classified  as  hits  for  the 
conventional  ROC  curve,  but  they  would  not  be  classified  as  correct  localizations  for  the  LROC  curve. 

In  short,  because  it  is  easier  to  respond  “target”  or  “nontarget”  than  to  specify  one  of  n  possible  locations, 
curves  such  as  the  LROC  fall  below  their  associated  ROC  curve. 


Figure  3:  Hypothetical  curves  for  detection  (ROC)  and  localization  (LROC). 

It  should  be  noted  that  curves  such  as  the  LROC  are  not  “true”  ROC  curves.  Unlike  ROC  curves 
for  detection,  LROC  curves  may  not  end  at  (1,  1).  The  endpoint  of  the  LROC  curve  is  derived  from  the 
proportion  of  correct  localization  judgments  across  all  target  images  and  will  therefore  end  at  (1,  1)  only 
when  localization  performance  is  perfect.  Consequently,  the  LROC  should  be  interpreted  with  caution. 

It  can  best  be  used  as  a  general  guideline  to  indicate,  for  example,  whether  localization  performance  is 
satisfactory  or  whether  additional  training  might  be  necessary  to  bridge  the  gap  between  detection  and 
localization. 
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The  Present  Study 


The  primary  purpose  of  the  present  study  was  to  quantify  the  effect  of  number  of  confusers  on 
operators’  detection  and  localization  performance,  confidence,  and  reaction  time  when  searching  for  a 
TEL  target  positioned  at  various  orientations  in  low,  medium,  and  high  clutter  backgrounds  with  2  ft  and 
4  ft  resolution  simulated  SAR  imagery.  As  such,  the  current  investigation  constitutes  an  incremental 
exploration  of  the  problem  of  TEL  target  detection.  In  previous  studies,  we  have  explored  the  detection 
and  localization  of  a  TEL  target  at  various  image  resolutions,  and  under  different  conditions  of  clutter 
(Davis,  See,  Shacklett,  &  Kuperman,  1996;  See,  Davis,  &  Kuperman,  1997;  See  &  Kuperman,  1995).  In 
this  study,  we  examined  the  added  effects  of  the  presence  of  TEL  confusers  on  TEL  target  detection  and 
localization. 

As  described  earlier,  previous  studies  have  demonstrated  that  the  presence  of  confusers  can  have 
a  detrimental  impact  on  search  performance,  producing  deteriorations  in  reaction  tim.e  and  detection 
probability  as  the  number  of  confusers  increases.  To  date,  however,  no  studies  have  concentrated  upon 
TEL  target  detection  in  the  presence  of  confusers.  Such  studies  have  also  not  applied  TSD  to  assess 
performance  effectiveness  in  terms  of  operator  sensitivity  and  bias  when  confusers  are  present.  Further, 
systematic  investigation  of  the  effects  of  orientation  has  not  yet  been  undertaken  with  ground  vehicles, 
let  alone  a  TEL  target.  The  goal  of  the  present  study  was  to  fill  these  voids.  We  expected  that 
performance  effectiveness  would  decline  1)  as  the  number  of  confusers  in  the  scene  increased,  2)  as  the 
orientation  deviated  from  a  90°  broadside  view  of  the  TEL,  3)  as  the  background  clutter  increased,  and  4) 
as  the  resolution  deteriorated  from  2  ft  to  4  ft. 


METHOD 

Participants 

The  participants  were  nine  airmen  and  one  civilian  from  various  organizations  at  Wright- 
Patterson  Air  Force  Base,  OH,  ranging  in  age  from  31  to  59  years  {M  =  42,  SD  =  9).  They  included 
Weapons  System  Operators  (WSO),  Instructor  WSOs  (IWSO),  Instructor  Radar  Navigators  (IRN),  Pilots, 
and  Instructor  Pilots.  They  had  flown  in  A- 10s,  B-52s,  C~141s,  F-4s,  F-15s,  F-16s,  and  F-1 1  Is.  Only  two 
of  the  observers  worked  with  SAR  imagery  as  part  of  their  current  job  assignment.  Of  the  remainder, 
four  had  previous  flight  experience  with  SAR  imagery.  The  remaining  individuals  had  participated  in 
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other  target  acquisition  studies  performed  in  this  lab  with  either  FLIR  or  SAR  imagery.  All  of  the 
operators  had  normal  or  corrected-to-normal  20/20  vision. 

Design 

The  design  was  a  2  (resolution)  x  2  (target  presence)  x  2  (scene  type)  x  3  (clutter)  x  4  (number  of 
confusers)  x  5  (target  orientation)  repeated  measures  design.  The  two  levels  of  SAR  image  resolution 
were  2  ft  and  4  ft.  Target  presence  referred  to  the  fact  that  the  target  was  not  present  in  all  of  the  images; 
some  images  contained  no  vehicles  at  all  (“empty”  scenes)  and  some  contained  only  the  confuser.  The 
three  levels  of  clutter  were  defined  according  to  the  number  of  trees  present  in  the  scene:  low  ( 1 5  trees), 
medium  (45  trees),  and  high  (135  trees).  Within  each  level  of  clutter,  two  different  tree  placement 
configurations  were  used  to  provide  the  two  levels  of  scene  type,  designated  simply  as  A  and  B.  The 
number  of  confusers  present  in  a  scene  was  either  0,  1,2,  or  3.  The  TEL  target  was  positioned  in 
orientations  of  0°  (head-on),  45®,  90®  (broadside),  135®,  and  180®  (tail-on). 

Apparatus 


Vehicles. 

The  two  vehicle  types  included  in  the  study  were  the  TEL  target  and  an  M-548  confuser.  The 
TEL  is  a  four  axle  eight-wheeled  vehicle  carrying  a  Scud  missile  that  can  be  launched  at  ground  targets. 
The  TEL  is  13.36  m  long  x  3.02  m  wide  x  3.5  m  high  and  weighs  37,400  kg  when  loaded  with  a  missile 
(“SS-I  ‘Scud,’”  1996).  The  missile  itself  is  1 1.16  m  long  and  is  carried  horizontally  on  the  TEL  until 
launch,  at  which  time  it  is  raised  to  the  vertical  position  at  the  back  of  the  TEL.  In  the  current  study,  the 
missile  was  always  carried  in  its  horizontal,  stowed  position  on  the  TEL. 

The  M-548  confuser  is  a  five-wheeled  tracked  cargo  carrier  used  in  the  U.  S.  Army  to  fulfill  a 
number  of  functions,  including  use  as  a  resupply  vehicle  for  self-propelled  artillery  units  equipped  with 
M 109  and  MHO  weapons.  The  engine  and  crew  compartment  are  at  the  front  of  the  vehicle.  The  cargo 
compartment  in  the  rear  of  the  vehicle  has  an  opening  in  the  back  with  two  doors.  The  cargo  area  can  be 
enclosed  using  a  standard  vinyl-coated  nylon  cover  supported  by  bo>vs.  A  ring  mount  on  top  of  the  cab 
may  be  fitted  with  a  7.62  mm  or  12.7  mm  machine  gun  for  air  and  local  defense  (as  it  was  in  the  present 
investigation).  A  winch  is  mounted  at  the  front  of  the  vehicle.  The  M-548  is  5.9  m  long  x  2.7  m  wide  x 
2.7  m  high  (“United  Defense,”  1996-97,  p.  504).  The  vehicle  has  been  employed  by  the  ATR 
development  community  as  a  confuser  in  the  testing  of  ATR  algorithms. 
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Stimuli, 

The  stimuli  consisted  of  simulated  SAR  imagery  created  with  X-Patch  ES  software  and 
computer-aided  design  (CAD)  models  of  the  TEL  and  M-548.  The  CAD  models  of  the  vehicles  were 
obtained  from  Veridian,  Inc.,  which  has  been  developing  a  SAR  database  under  contract  to  the  Target 
Recognition  Branch,  Sensor  and  ATR  Technology  Division,  Sensor  Directorate,  of  the  Air  Force 
Research  Laboratory  (AFRL/SNAT),  Model-Based  Vision  Laboratory  (MBVLAB).  The  database, 
which  can  be  accessed  through  the  laboratory’s  home  page  at  http://www.mbvlab.wpafb.af.mil/internal/ 
(though  some  sections  are  password-protected),  is  envisioned  as  a  long-term  repository  for  various  forms 
of  sensor  data  and  associated  information  such  as  maps,  photographs,  algorithms,  and  performance 
results.  One  of  the  projects  currently  included  in  the  database,  Moving  and  Stationary  Target 
Recognition  (MSTAR),  has  involved  the  collection  of  SAR  imagery  of  a  variety  of  vehicles,  including 
the  TEL  and  the  M-548  cargo  truck.  As  a  result  of  this  image  collection  effort,  we  were  able  to  obtain 
CAD  models  of  the  TEL  and  its  designated  confuser  vehicle,  the  M-548. 

The  CAD  models  of  the  vehicles  were  used  in  conjunction  with  the  X-Patch  ES  SAR  simulation 
software  to  create  the  SAR  imagery  needed  to  fulfill  the  experimental  design  objectives.  The  X-Patch  ES 
software  enables  one  to  develop  any  of  a  number  of  different  backgrounds  into  which  various  types  of 
targets  can  then  be  embedded.  In  addition,  the  programmer  can  control  such  factors  as  grazing  angle, 
image  resolution,  and  target  location,  making  it  possible  to  generate  an  almost  endless  variety  of 
simulated  SAR  images.  The  end  result  is  a  set  of  images  that  may  be  much  more  realistic  in  appearance 
than  what  can  be  achieved  with  other  simulation  packages.  For  the  current  study,  we  began  by 
generating  the  low,  medium,  and  high  clutter  backgrounds  at  two  different  image  resolutions  (2  ft  and  4 
ft)  and  a  constant  grazing  angle  of  15®  (grazing  angle  is  the  angle  formed  between  the  ground  and  the  line 
of  sight  from  the  sensor  to  the  target).  The  backgrounds  were  developed  by  altering  the  tree  density  in 
the  scene;  tree  diameter  and  height  remained  constant.  The  number  of  trees  in  the  low,  medium,  and  high 
clutter  backgrounds  were  15, 45,  and  135,  respectively.  Within  each  Resolution  x  Clutter  condition,  two 
different  scenes  (referred  to  as  A  and  B)  were  created  by  varying  the  placement  of  the  trees  in  the  image. 
This  was  done  to  introduce  some  variety  in  the  image  set  and  avoid  inordinate  repetition  of  the  same 
scene  during  the  experimental  session.  The  2  (resolution)  x  2  (scene  type)  x  3  (clutter)  combination 
provided  the  base  set  of  12  empty  scenes  (containing  no  targets  or  confusers)  used  in  the  study. 

Images  with  vehicles  were  created  by  embedding  the  CAD  models  of  the  TEL  and  M-548  into 
the  X-Patch  ES  input  data  files  scenes.  Each  simulated  SAR  output  image  contained  either  no  vehicles, 
one  confuser,  one  TEL,  or  one  TEL  plus  one,  two,  or  three  confusers.  Hence,  the  maximum  number  of 
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vehicles  per  image  was  four.  For  this  reason,  vehicle  placement  was  determined  by  dividing  the  image 
into  quadrants  and  placing  no  more  than  one  vehicle  randomly  within  each  quadrant.  The  vehicles  were 
positioned  in  orientations  of  either  0°  (head-on),  45°,  90°  (broadside),  135°,  or  180°  (tail-on).  In  any 
given  image,  the  orientation  of  the  TEL  target  represented  the  independent  variable  of  concern.  The 
orientations  of  the  confusers,  when  present,  were  allowed  to  vary  randomly,  within  the  constraints 
imposed  by  the  five  pre-selected  levels  of  orientation. 

Quadrant  assignment  for  each  vehicle  was  determined  by  first  dividing  the  images  into  five  .sets 
according  to  the  number  and  type  of  vehicles  present:  1)  target  only,  2)  confuser  only,  3)  target  plus  one 
confuser,  4)  target  plus  two  confusers,  and  5)  target  plus  three  confusers.  Within  each  set,  30  images 
needed  to  be  generated  to- produce  the  required  2  (resolution)  x  3  (clutter)  x  5  (target  orientations)  image 
set.  This  procedure  resulted  in  the  Scene  A  images.  Once  vehicle  placement  for  the  30  images  within 
each  set  had  been  determined  for  Scene  A,  they  were  duplicated  against  the  Scene  B  background.  For  the 
target-only  and  confuser-only  images,  the  basic  strategy  was  to  represent  each  combination  of  vehicle 
orientation  and  quadrant  location  at  least  once  in  the  set  of  30  images.  Within  the  quadrant,  the  vehicle 
was  placed  randomly  in  an  open  area  where  it  would  not  be  obscured  by  trees  or  tree  shadows.  For  the 
target-plus-confuser  images,  the  most  critical  objective  was  to  represent  each  combination  of  target 
orientation  and  quadrant  location  at  least  once  since  the  orientation  of  the  target  was  the  experimental 
manipulation  of  concern.  Once  that  had  been  accomplished,  the  intent  was  to  represent  as  many 
combinations  of  target  orientation,  target  location,  confuser  orientation,  and  confuser  location  as  possible 
in  each  set  of  30  images.  The  vehicles  were  always  placed  in  separate  quadrants,  but  they  could  be 
located  anywhere  from  66  pixels  to  402  pixels  of  each  other  {M  =  202,  SD  =  62).  The  resulting  images 
were  512  pixels  by  512  pixels  in  size  and  were  displayed  using  the  green  gun  of  the  monitor.  Including 
both  the  6  empty  background  scenes  and  the  150  images  containing  vehicles,  the  final  image  set 
consisted  of  156  Scene  A  images  and  156  Scene  B  images,  for  a  total  of  312  unique  SAR  images. 

A  similar  procedure  was  used  to  generate  12  unique  practice  images,  designed  to  familiarize  the 
participants  not  only  with  the  appearance  of  the  radar  returns  for  the  TEL  and  M-548  but  also  with  the 
procedures  to  be  followed  for  each  image  presentation.  Of  the  practice  images,  six  contained  a  single 
TEL  target  and  six  contained  a  single  M-548  confuser.  The  six  images  consisted  of  low,  medium,  and 
high  clutter  backgrounds  at  both  the  2  ft  and  4  ft  resolutions;  half  of  these  were  Scene  A  images  and  half 
were  Scene  B.  Further,  each  vehicle  was  positioned  at  least  once  in  each  of  the  five  possible  orientations 
and  in  each  of  the  four  possible  quadrants  of  the  display.  Thus,  by  the  time  the  experimental  session 
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began,  the  participants  had  seen  both  of  the  vehicles  in  all  of  the  potential  image  resolutions,  clutter 
backgrounds,  orientations,  and  quadrants  in  which  they  would  subsequently  appear. 

During  the  experimental  session,  the  imagery  was  presented  in  5  blocks  of  3 12  images,  for  a  total 
of  1560  images.  The  312  images  consisted  of  156  Scene  A  images  (78  at  the  2  ft  resolution  and  78  at  4 
ft)  and  156  Scene  B  images  (78  at  2  ft  and  78  at  4  ft).  Within  each  block,  the  images  were  presented  in 
sub-blocks  based  on  resolution;  within  sub-blocks,  the  presentation  order  was  random.  Half  of  the 
participants  began  each  block  with  the  2  ft  resolution,  and  the  other  half  began  each  block  with  the  4  ft 
resolution.  Across  blocks,  the  images  in  each  set  were  identical  (though  they  were  presented  in  a 
different  order  each  time).  Replicates  were  presented  to  permit  the  derivation  of  reliable  estimates  of  hit 
and  false  alarm  percentages  for  the  subsequent  calculation  of  sensitivity  and  response  bias  as  well  as  the 
generation  of  ROC  curves.  The  replicates  were  presented  in  five  blocks  for  three  reasons.  First, 
presenting  replicates  across  blocks  rather  than  within  a  block  would  decrease  the  likelihood  that 
participants  would  base  their  decisions  on  recall  of  their  responses  during  earlier  trials  (i.e.,  it  would  be 
harder  to  recall  across  blocks  as  opposed  to  within  blocks).  Second,  this  presentation  format  would  allow 
an  assessment  of  potential  practice  effects  on  performance.  Third,  at  the  completion  of  a  block  of  trials, 
each  participant  could  individually  decide  whether  to  complete  additional  blocks  or  return  at  a  later  time. 
It  was  expected  that  scene  type  would  not  have  a  significant  effect  on  performance  and  would  therefore 
permit  the  data  to  be  collapsed  across  this  variable,  thereby  producing  ten  replicates  of  each  image 
instead  of  five. 


Equipment. 

Data  collection  occurred  in  the  Crew  Aiding  and  Information  Warfare  Analysis  Laboratory 
(CIWAL)  located  in  the  Air  Force  Research  Laboratory  at  Wright-Patterson  Air  Force  Base,  OH.  A 
Silicon  Graphics  02  computer  was  used  for  stimulus  presentation  and  data  collection.  The  computer 
monitor  was  placed  on  a  table  slightly  below  eye  level,  and  participants  sat  approximately  70  cm  from  it. 
They  used  a  keypad  placed  in  front  of  the  monitor  to  indicate  their  target/nontarget  decision  and 
confidence  for  each  image  and  a  computer  mouse  to  designate  the  location  of  the  TEL  target  if  they 
believed  it  to  be  present.  Prior  to  the  experiment,  the  brightness  and  contrast  controls  of  the  monitor 
were  adjusted  to  clearly  display  each  step  of  a  16-step  gray  scale  without  oversaturating  the  monitor.  A 
TOPCON  BM-7  luminance  colorimeter  was  used  to  measure  the  luminance  of  the  100%  and  50%  white 
fields  from  the  gray  scale  each  week  for  the  duration  of  the  study.  The  luminance  of  the  100%  white 
field  varied  from  40.9  cd/m"  to  42.1  cdW  (A/  =  41 .6,  5D  =  0.6),  and  the  luminance  of  the  50%  white 
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field  ranged  from  1 1 .3  cd/m‘  to  1 1 .7  cd/m"  (M  =  1 1 .5,  SD  =  0.2).  Participants  were  instructed  not  to 
adjust  the  brightness  and  contrast  controls  during  data  collection.  The  overhead  lights  in  the  room  were 
dimmed  during  the  experimental  session  so  as  to  minimize  glare  on  the  monitor  while  providing 
sufficient  light  for  participants  to  complete  their  task. 

Procedure 

When  individuals  reported  to  the  CIWAL  facility  to  participate  in  the  study,  the  experimenter 
first  gave  them  a  brief  oral  description  of  the  experiment  and  then  asked  them  to  read  a  standard  consent 
form  and  sign  it  if  they  wished  to  participate.  After  the  individual  signed  the  consent  form  and  responded 
to  a  general  background  questionnaire,  the  experimenter  presented  a  more  detailed  briefing  of  the  study, 
which  included  a  description  of  the  TEL  and  M-548  vehicles  and  the  various  resolutions,  orientations, 
and  scene  backgrounds  in  which  they  would  appear.  The  experimenter  provided  SAR  images  of  each 
vehicle  in  each  of  its  five  orientations  at  both  the  2  ft  and  4  ft  re.solutions  for  the  participant  to  study. 
These  images  were  also  available  for  review  between  blocks. 

Following  the  briefing,  the  experimenter  tested  the  individual’s  vision  using  a  standard  Snellen 
eye  chart  and  then  escorted  the  participant  to  the  room  where  the  task  would  be  completed.  The  lights 
were  dimmed  and  once  the  individual  was  seated  comfortably  in  front  of  the  computer,  the  experimenter 
described  the  procedure  to  be  followed  during  each  image  presentation  in  both  the  practice  and 
experimental  sessions.  A  READY  prompt  first  appeared  in  the  center  of  a  black  background  on  the 
computer  screen.  When  ready  to  begin  the  trial,  the  participant  pressed  a  labeled  READY  button  on  the 
keypad  to  bring  up  a  SAR  image.  Following  the  appearance  of  the  image,  the  observer  first  had  to  decide 
whether  or  not  the  TEL  target  was  present  in  the  image  or  not  and  simultaneously  rate  his/her  confidence 
in  that  decision.  Once  the  decision  had  been  reached,  the  operator  pressed  one  of  six  labeled  buttons  on 
the  keypad:  (1)  target  definitely  not  present,  (2)  target  probably  not  present,  (3)  target  possibly  not 
present,  (4)  target  possibly  present,  (5)  target  probably  present,  and  (6)  target  definitely  present.  Thus,  a 
response  of  1, 2,  or  3  signified  that  the  operator  thought  a  target  was  not  present  in  the  image,  whereas  a 
response  of  4,  5,  or  6  corresponded  to  a  decision  that  the  target  was  present  in  the  image.  Responses  of 
“definitely,”  “probably,”  and  “possibly”  represented  variations  in  the  operator’s  confidence  in  the 
detection  decision,  ranging  from  high  to  low  confidence.  If  the  operator  decided  that  the  target  was  not 
present,  the  image  disappeared  from  the  screen  and  was  replaced  by  the  READY  prompt  for  the  next 
image.  If  the  operator  thought  the  TEL  target  was  present,  the  image  remained  on  the  screen  so  that  he 
could  designate  the  location  of  the  target  in  the  scene.  A  message  instructing  the  observer  to  designate 
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the  location  of  the  TEL  with  the  mouse  pointer  appeared  beside  the  image.  Once  the  localization 
response  had  been  made,  the  image  disappeared  from  the  screen  and  the  READY  prompt  for  the  next 
image  appeared.  This  sequence,  which  is  depicted  in  Figure  4,  was  repeated  until  all  312  images  in  the 
block  had  been  completed.  At  this  point,  the  operator  notified  the  experimenter  that  the  block  was  over 
and  decided  whether  to  continue  with  additional  blocks  or  return  at  a  later  time  to  complete  the  study. 

Six  of  the  participants  completed  all  five  blocks  in  one  session;  two  participants  completed  the  blocks  in 
two  separate  sessions  on  the  same  day;  and  the  remaining  two  operators  finished  the  experiment  in  two 
separate  sessions  over  a  two  day  period. 

A  practice  session  with  12  images  preceded  the  first  block  of  experimental  trials.  The  purpose  of 
the  practice  session  was  to  familiarize  the  observers  with  the  nature  of  the  task  and  the  appearance  of  the 
vehicles  in  the  SAR  imagery.  The  experimenter  was  present  during  the  practice  session  to  answer  any 
questions  and  provide  performance  feedback  when  needed  but  left  the  room  when  the  individual  was 
ready  to  begin  a  block  of  experimental  trials.  The  operators  were  encouraged  to  take  breaks  between 
images  as  needed.  Further,  in  between  the  sub-blocks  of  2  ft  and  4  ft  resolution  images,  a  message 
appeared  on  the  screen  instructing  the  operator  to  take  a  break.  Individuals  who  completed  multiple 
blocks  at  once  were  also  permitted  to  take  a  break  between  blocks  while  the  experimenter  made  the 
necessary  preparations  on  the  computer.  The  duration  of  each  block  ranged  from  13  to  53  minutes  (M  = 
25,  SD  =  9). 
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Figure  4:  The  sequence  of  events  during  each  trial. 

RESULTS 

The  dependent  variables  that  were  analyzed  in  the  present  study  included  proportions  of  hits  and 
false  alarms;  the  signal  detection  theory  measures  of  perceptual  sensitivity  and  response  bias;  reaction 
times  (RTs)  for  hits,  false  alarms,  misses,  and  correct  rejections;  and  operator  confidence.  In  most  cases, 
2  (resolution)  x  3  (clutter)  x  4  (number  of  confusers)  x  5  (vehicle  orientation)  repeated-measures  analyses 
of  variance  (ANOVAs)  were  used  to  test  the  statistical  significance  of  the  means.  The  alpha  level  for  all 
ANOVAs  was  set  at  .05.  Probabilities  for  any  effect  containing  three  or  more  levels  (e.g.,  clutter, 
number  of  confusers,  and  orientation)  were  obtained  via  the  Huynh-Feldt  epsilon  adjustment  to  degrees 
of  freedom  (Huynh  &  Feldt,  1970,  1976).  Where  needed,  post  hoc  analyses  in  the  form  of  correlated  /- 
tests  were  conducted.  For  the  purposes  of  this  study,  any  comparison  with  a  probability  of  .01  or  less 
was  considered  statistically  significant. 


Hits  and  False  Alarms 

The  derivation  of  hits  and  false  alarms  in  the  current  study  represents  a  departure  from  the 
procedures  we  used  in  a  similar  study  where  participants  were  also  asked  to  make  both  detection  and 


24 


localization  responses  (Davis,  See,  Shacklett,  &  Kuperman,  1996).  In  the  1996  study,  participants 
viewed  SAR  patch  maps  containing  either  empty  scenes  with  no  vehicles  present  or  scenes  containing  the 
TEL  target  to  be  detected  plus  two  additional  support  vehicles.  As  in  the  present  study,  their  task  was  to 
determine  whether  the  TEL  was  present  in  the  image  (detection)  and  to  designate  its  location  if  they 
believed  it  to  be  present  (localization).  In  the  1996  study,  a  hit  was  defined  as  a  “detection”  response  to  a 
target  image,  and  a  false  alarm  was  defined  as  a  “detection”  response  to  an  empty  scene.  The  correct 
detections  were  further  identified  as  either  correct  or  incorrect  localizations  of  the  TEL  vehicle,  and  the 
signal  detection  theory  measures  of  sensitivity  and  bias  were  computed  for  both  detection  and 
localization.  With  this  type  of  procedure,  a  “correct  detection”  response  to  a  target  image  will  be 
classified  as  a  hit,  regardless  of  whether  the  localization  response  itself  is  correct  or  not.  Thus,  in  some 
instances,  the  localization  response  may  indicate  that  the  “hit”  resulted  not  from  correct  identification  of 
the  target  but  from  inspection  of  some  other  object  that  the  operator  believed  to  be  the  target,  an  object 
that  may  even  be  far  removed  from  the  actual  location  of  the  TEL.  One  may  then  legitimately  ask 
whether  the  detection  response  should  have  been  classified  as  correct  in  the  first  place.  Did  the  observer 
truly  “detect”  the  target? 

To  circumvent  these  difficulties  in  the  current  investigation,  we  defined  a  hit  as  a  keypad 
response  of  4  (target  possibly  present),  5  (target  probably  present),  or  6  (target  definitely  present)  to  a 
target  image  that  was  subsequently  accompanied  by  a  correct  localization  of  the  TEL  vehicle.  If  the 
localization  was  incorrect,  the  response  was  considered  simultaneously  to  be  a  miss  (since  the  operator 
did  not  locate  the  TEL  target)  and  a  false  alarm  (since  the  operator  incorrectly  chose  some  other  object  as 
the  TEL).  If  the  operator  initially  responded  with  a  1  (target  definitely  not  present),  2  (target  probably 
not  present),  or  3  (target  possibly  not  present)  to  a  target  image,  that  response  was  considered  a  miss 
only.  Thus,  each  target  image  produced  both  a  proportion  of  hits  and  a  proportion  of  false  alarms. 

Recall  also  that  the  image  set  contained  some  images  with  only  a  single  M-548  confuser  present.  In 
accordance  with  the  traditional  applications  of  signal  detection  theory,  these  images  produced  either 
correct  rejections  (responses  of  1,  2,  or  3)  or  false  alarms  (responses  of  4,  5,  or  6).  The  false  alarms  for 
these  images  were  used  to  establish  a  baseline  for  the  number  of  false  alarms  that  might  be  expected  with 
images  containing  2,  3,  or  4  confusers  (since  these  were  not  present  in  the  actual  image  set).  Within  each 
experimental  condition,  the  false  alarms  associated  with  the  target  images  were  then  added  onto  this 
baseline  to  derive  the  total  proportion  of  false  alarms  in  each  condition.  Technically,  the  empty 
background  scenes  in  the  experiment  could  also  produce  either  correct  rejections  or  false  alarms. 
However,  since  none  of  the  operators  made  any  false  alarms  with  these  images,  they  were  not  included  in 
the  totals. 
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In  essence,  this  procedure  combines  the  operator’s  detection  response  and  localization  response 
to  target  images  to  determine  whether  the  decision  is  a  hit  or  a  false  alarm.  In  contrast  to  our  1996  study, 
the  detection  and  localization  responses  will  not  be  subjected  to  separate  analyses.  In  this  case,  a  hit  can 
be  considered  a  “true”  hit  since  the  operator  not  only  detected  the  target  but  also  correctly  selected  it 
from  all  the  other  objects  and  vehicles  present  in  the  scene. 

Preliminary  Analyses. 

Prior  to  conducting  the  full  factorial  analyses  of  hits  and  fal.se  alarms  in  the  present  study,  several 
preliminary  inspections  of  the  data  were  completed.  First,  in  an  effort  to  verify  that  none  of  the  images  in 
the  image  set  was  an  “outlier,”  the  hits  and  false  alarms  associated  with  each  unique  image  were 
examined.  Only  3%  of  the  images  were  associated  with  hits  below  90%  (the  minimum  was  68%),  and 
only  1%  produced  false  alarms  of  10%  or  greater  (the  maximum  was  30%).  Because  visual  inspection  of 
these  images  indicated  no  anomalies,  none  of  the  images  was  dropped  from  the  image  set. 

Second,  in  order  to  determine  whether  the  presentation  order  of  image  resolution  affected 
operator  performance,  /-tests  of  both  the  hits  and  false  alarms  were  conducted.  All  operators  saw  both 
levels  of  image  resolution,  but  half  of  them  always  evaluated  the  2  ft  resolution  first  (the  “24”  group)  and 
the  other  half  always  saw  the  4  ft  resolution  first  (the  “42”  group).  Mean  proportions  of  hits  were  .98 
{SD  =  .02)  for  the  “24”  group  and  .95  (SD  =  .05)  for  the  “42”  group,  a  difference  that  was  not  statistically 
significant,  /  (8)  =  .9,  p  >  .05.  Mean  proportions  of  false  alarms  for  the  two  groups  were  .01  {SD  =  .003) 
and  .02  {SD  =  .013),  respectively.  As  with  the  hits,  this  difference  was  not  significant,  /  (8)  =  2.2,  p  > 

.05. 


Third,  a  test  of  the  effects  of  scene  type  was  completed.  Recall  that  two  versions  of  each  image 
were  generated,  differing  only  in  the  placement  of  the  trees  and  vehicles  within  the  scene.  Changes  in 
scene  type  were  used  to  produce  variety  in  the  image  set  so  the  images  would  not  simply  appear  to  be 
repetitions  of  each  other.  Thus,  scene  type  was  not  expected  to  impact  operator  performance.  Neither 
the  difference  in  hits  between  the  “A”  and  “B”  scenes  {M  =  .00017)  nor  the  difference  in  false  alarms  (M 
=  .00047)  was  statistically  significant,  p  >  .05. 

Finally,  the  effects  of  block  were  examined  to  study  changes  in  performance  with  repeated 
practice.  A  one-way  ANOVA  of  the  proportion  of  hits  was  statistically  significant,  F  (4,36)  =  5.0,  p  < 
.04.  The  hits  tended  to  increase  over  blocks  from  a  mean  of  .90  {SD  =  .12)  in  the  first  block  to  a  mean  of 
.99  {SD  =  .01)  in  the  fifth  block.  However,  post  hoc  testing  indicated  that  none  of  the  differences  in 
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means  was  statistically  significant,  /?  >  .01 .  With  respect  to  the  analysis  of  false  alarms,  mean  false 
alarms  were  .03  (SD  =  .03)  in  the  first  block  and  only  .004  (SD  =  .003)  in  the  fifth  block.  The  drop  in 
false  alarms  over  block  was  not  significant,  F  (4,36)  =  3.5,  p>  .05. 


Hits, 

Following  the  preliminary  inspections  of  the  data,  analysis  of  the  four  primary  independent 
variables  was  begun.  Mean  proportions  of  hits  in  each  condition  of  resolution,  background  clutter, 
number  of  confusers,  and  vehicle  orientation  are  portrayed  in  Table  1 .  As  can  be  seen  in  the  table,  the 
overall  proportion  of  hits  in  the  study  was  quite  high  and  did  not  deviate  greatly  from  this  level, 
indicating  that  the  operators  seldom  mistook  the  confuser  for  the  TEL.  (Note:  inspection  of  the  data 
revealed  that  on  the  TEL  trials  where  operators  indicated  that  a  target  was  definitely,  probably,  or 
possibly  present,  they  correctly  located  the  TEL  99.3%  of  the  time.  They  erroneously  chose  the  confuser 
only  0.5%  of  the  time  and  some  other  object  in  the  background  a  mere  0.2%  of  the  time.)  In  fact,  the 
proportion  of  hits  was  identical  at  the  2  ft  and  4  ft  resolutions  and  was  nearly  constant  as  the  number  of 
confusers  increased  from  0  to  3.  Within  each  resolution,  there  did  appear  to  be  some  variation  as  a  result 
of  vehicle  orientation.  Orientations  of  0°  and  180°  tended  to  be  associated  with  the  lowest  proportions  of 
hits,  whereas  the  hits  for  orientations  of  45°,  90°,  and  135°  were  more  or  less  similar  to  one  another.  A 
summary  of  the  ANOVA  of  the  means  in  Table  1  can  be  found  in  Table  2.  In  accordance  with 
observations  of  the  means  in  Table  1 ,  none  of  the  main  effects  of  resolution,  clutter,  number  of  confusers, 
and  orientation  was  statistically  significant.  However,  vehicle  orientation  did  enter  into  several 
interactions. 
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MEAN 


Table  2.  Summary  of  a  2  (resolution)  x  3  (clutter)  x  4  (no.  confusers)  x  5  (orientation)  ANOVA  of 
Hits. 


Source 

df 

MS 

F 

p 

Resolution 

1,9 

.006 

<  1 

NS 

Clutter 

2,  18 

.013 

3.8 

NS 

No.  Confusers 

3,27 

.006 

2.3 

NS 

Orientation 

4,  36 

.087 

3.1 

NS 

Resolution  x  Clutter 

2,  18 

.006 

6.7 

.008 

Resolution  x  No.  Confusers 

3,27 

.001 

<  1 

NS 

Resolution  x  Orientation 

4,  36 

.013 

<  1 

NS 

Clutter  X  No.  Confusers 

6,  54 

.003 

2.4 

.04 

Clutter  X  Orientation 

8,72 

.004 

3.2 

.006 

No.  Confusers  x  Orientation 

12,  108 

.004 

1.7 

NS 

Resolution  x  Clutter  x  No.  Confusers 

6,  54 

.004 

1.5 

NS 

Resolution  x  Clutter  x  Orientation 

8,72 

.004 

2.2 

NS 

Resolution  x  No.  Confusers  x  Orientation 

12,  108 

.006 

4.0 

.002 

Clutter  X  No.  Confusers  x  Orientation 

24,216 

.006 

4.9 

.0001 

Resolution  x  Clutter  x  No.  Confusers  x  Orientation 

24,216 

.005 

2.8 

.008 

The  Resolution  x  Clutter  interaction  is  depicted  graphically  in  Figure  5.  As  can  be  seen  in  the 
figure,  the  proportion  of  hits  was  slightly  higher  at  the  2  ft  resolution  as  compared  to  4  ft  in  the  low  and 
medium  clutter  backgrounds;  in  the  high  clutter  background,  the  trend  inexplicably  reversed  itself.  This 
reversal  may  have  been  due  to  the  fact  that  clutter  was  defined  as  the  number  of  trees  per  image  rather 
than  per  area.  Thus,  since  the  image  size  remained  constant,  the  same  number  of  trees  occupied  what 
appeared  to  be  a  larger  area  at  the  4  ft  resolution  than  was  true  for  the  2  ft  resolution.  Hence,  the  high 
clutter  background  was  extremely  cluttered  at  the  2  ft  resolution  but  much  more  open  at  the  4  ft 
resolution,  making  it  relatively  easier  to  detect  vehicles  in  the  open  spaces  when  the  resolution  was 
poorer.  While  the  same  number  of  trees  (clutter  objects)  appeared  in  the  high  clutter  background  at  both 
resolutions,  the  low  resolution  imagery  actually  appeared  visually  to  be  less  cluttered.  Post  hoc 
correlated  Mests  were  conducted  to  determine  whether  the  hits  differed  between  the  2  ft  and  4  ft 
resolutions  within  each  level  of  background  clutter.  None  of  the  differences  was  statistically  significant 
(p>my 
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Background  Clutter 


Figure  5:  Mean  proportion  of  hits  at  the  2  ft  and  4  ft  resolutions  in  the  low  (L),  medium  (M),  and 
high  (H)  clutter  backgrounds. 

The  interaction  between  clutter  and  number  of  confusers  is  portrayed  in  Figure  6.  The  figure 
reveals  that  the  proportion  of  hits  was  relatively  constant  as  the  number  of  confusers  increased  in  the  low 
and  medium  clutter  backgrounds.  However,  in  the  high  clutter  background,  the  proportion  of  hits 
dropped  noticeably  when  two  confusers  were  present  before  returning  to  the  level  common  to  the  other 
conditions.  A  comparison  of  the  low,  medium,  and  high  clutter  backgrounds  at  each  number  of  confusers 
indicated  that  the  only  significant  difference  occurred  between  the  low  and  high  backgrounds  when  two 
confusers  were  present  {p  <  .003).  In  that  condition,  a  lower  proportion  of  hits  occurred  in  the  high 
clutter  background  as  compared  to  low  clutter. 


Figure  6:  Mean  proportion  of  hits  as  a  function  of  number  of  confusers  for  the  low  (L),  medium 
(M),  and  high  (H)  clutter  backgrounds. 

The  nature  of  the  Clutter  x  Orientation  interaction  can  be  readily  seen  in  the  distinctive  pattern  of 
the  plot  in  Figure  7.  Namely,  differences  among  the  three  levels  of  clutter  were  apparent  only  when  the 
orientation  was  either  (f  or  180"^;  otherwise,  the  proportions  of  hits  in  the  low,  medium,  and  high  clutter 
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backgrounds  were  nearly  identical.  Post  hoc  tests  indicated  that  the  only  significant  difference  was 
between  the  low  and  high  clutter  backgrounds  at  an  orientation  ofO^(p<  .004). 


Figure  7:  Mean  proportion  of  hits  as  a  function  of  vehicle  orientation  for  the  low  (L),  medium  (M), 
and  high  (H)  clutter  backgrounds. 

The  Resolution  x  Number  of  Confusers  x  Orientation  interaction  is  portrayed  in  the  two  plots  of 
Figure  8.  The  first  plot  shows  the  proportion  of  hits  as  a  function  of  vehicle  orientation  for  each  level  of 
confusers  at  the  2  ft  resolution.  The  second  plot  shows  the  same  relationship  at  the  4  ft  resolution. 
Inspection  of  the  two  plots  reveals  that  the  interaction  appeared  to  stem  primarily  from  the  2  ft  resolution. 
At  the  4  ft  resolution,  there  was  very  little  variation  among  the  levels  of  confusers  as  the  orientation 
changed  from  0°  to  180"^.  The  same  can  be  said  of  the  2  ft  resolution  for  all  orientations  except  180°, 
where  the  proportion  of  hits  v/as  highest  when  three  confusers  were  present  and  lowest  when  two 
confusers  appeared  in  the  scene.  To  determine  where  the  significant  differences  lay,  post  hoc  correlated 
Mests  were  conducted.  Within  each  resolution,  the  proportions  of  hits  associated  with  0,  1,2,  and  3 
confusers  were  compared  within  each  level  of  vehicle  orientation.  The  only  significant  difference 
occurred  at  an  orientation  of  180°  in  the  2  ft  resolution — the  proportion  of  hits  was  significantly  greater 
when  three  confusers  were  present  as  opposed  to  when  only  two  were  present  (p  <  .0004). 
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Figure  8:  Mean  proportion  of  hits  as  a  function  of  vehicle  orientation  for  each  number  of 
confusers  at  the  2  ft  (left)  and  4  ft  (right)  resolutions. 

The  three  plots  in  Figure  9  depict  the  three-way  interaction  between  clutter,  number  of  confusers, 
and  orientation.  The  first  plot  shows  the  proportion  of  hits  as  a  function  of  vehicle  orientation  for  each 
level  of  confusers  in  the  low  clutter  background.  The  second  plot  shows  the  same  relationship  in  the 
medium  clutter  background,  and  the  third  portrays  high  clutter.  The  three  plots  reveal  that  the  variation 
in  detection  performance  among  the  four  levels  of  confusers  tended  to  increase  as  the  background  clutter 
increased,  particularly  at  the  lowest  and  highest  vehicle  orientations.  Post  hoc  testing  indicated  that  the 
proportions  of  hits  associated  with  each  number  of  confusers  within  each  vehicle  orientation  did  not 
differ  significantly  from  one  another  in  the  low  and  medium  clutter  backgrounds  (/?  >  .01).  In  the  high 
clutter  background,  there  were  significant  differences  between  one  and  two  confusers  (p  <  .005)  and 
between  two  and  three  confusers  {p  <  .003)  when  the  orientation  was  180°.  The  proportions  of  hits  were 
significantly  greater  when  one  confuser  or  three  confusers  were  present  as  opposed  to  when  two 
confusers  appeared  in  the  scene. 
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Figure  9:  Mean  proportion  of  hits  as  a  function  of  vehicle  orientation  for  each  number  of 
confusers  in  the  low  (right),  medium  (left),  and  high  (bottom  center)  clutter  backgrounds. 

Finally,  in  the  analysis  of  hits,  the  four-way  interaction  between  resolution,  clutter,  number  of 
confusers,  and  orientation  was  also  statistically  significant.  The  nature  of  this  interaction  is  portrayed  in 
the  six  plots  of  Figure  10.  The  first  three  plots  depict  the  proportion  of  hits  as  a  function  of  vehicle 
orientation  for  each  number  of  confusers  in  the  low,  medium,  and  high  clutter  backgrounds  at  the  2  ft 
resolution.  The  last  three  plots  show  the  same  relationships  for  the  low,  medium,  and  high  clutter 
backgrounds  at  the  4  ft  resolution.  At  the  2  ft  resolution,  minor  fluctuations  in  hits  among  the  four  levels 
of  confusers  occurred  at  each  vehicle  orientation  when  the  clutter  was  low  and  medium.  In  the  high 
clutter  background,  performance  variation  was  primarily  limited  to  the  highest  vehicle  orientation.  At  the 
4  ft  resolution,  performance  variation  was  more  or  less  confined  to  the  lowest  and  highest  orientations  in 
the  medium  and  high  clutter  backgrounds.  Post  hoc  correlated  Mests  revealed  that,  at  the  2  ft  resolution, 
the  only  significant  difference  occurred  in  the  high  clutter  background  at  an  orientation  of  1 80° — the 
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proportion  of  hits  was  greater  when  three  confusers  were  present  as  opposed  to  two  (p  <  .0012).  At  the  4 
ft  resolution,  none  of  the  differences  was  statistically  significant  (p  >  .01). 
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Figure  10:  Mean  proportion  of  hits  as  a  function  of  vehicle  orientation  for  each  number  of 
confusers  at  each  image  resolution  and  clutter  level. 
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False  Alarms. 

Mean  proportions  of  false  alarms  in  each  condition  of  resolution,  background  clutter,  number  of 
confusers,  and  vehicle  orientation  are  portrayed  in  Table  3.  The  figures  in  the  table  reveal  that  the 
overall  percentage  of  false  alarms  in  the  study  was  only  4%.  The  false  alarms  were  slightly  lower  at  the 
2  ft  resolution  as  compared  to  4  ft.  In  addition,  they  were  slightly  lower  when  0  or  1  confusers  were 
present  as  opposed  to  when  2  or  3  confusers  appeared  in  an  image.  Within  each  level  of  resolution,  the 
false  alarms  tended  to  be  higher  at  orientations  of  0°  and  180°  as  well.  A  summary  of  the  ANOVA  of  the 
means  in  Table  3  appears  in  Table  4.  The  only  significant  main  effects  were  for  clutter  and  orientation. 
Further,  orientation  entered  into  a  number  of  two-way  and  three-way  interactions.  With  respect  to  the 
main  effect  for  clutter,  the  proportion  of  false  alarms  was  somewhat  higher  in  the  high  clutter  background 
(A/  =  .04,  SD  =  .01)  as  opposed  to  low  (M  =  .03,  5D  =  .01)  and  medium  (M  =  .03,  SD  =  .01).  However, 
post-hoc  testing  revealed  that  none  of  the  differences  was  significant,  p  >  .01 .  With  regard  to  the  main 
effect  for  orientation,  the  proportions  of  false  alarms  were  higher  at  orientations  of  0°  {M  =  .04,  SD  =  .02) 
and  1 80°  (M  =  .05,  SD  =  .03)  as  compared  to  45°  (M  =  .03,  SD  =  .003),  90°  (M  =  .03,  SD  =  .004),  and 
135°  (M  =  .03,  SD  =  .004).  Post  hoc  testing  indicated  that  none  of  the  differences  attained  statistical 
significance  at  the  .01  level. 
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Table  3.  Mean  Proportions  of  False  Alarms  (Standard  Deviations  in  Parentheses)  in  Each  Condition. 
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MEAN 


Table  4.  Summary  of  a  2  (resolution)  x  3  (clutter)  x  4  (no.  confusers)  x  5  (orientation)  ANOVA  of 
False  Alarms. 


Source 

df 

MS 

F 

p 

Resolution 

1,9 

.0003 

<  1 

NS 

Clutter 

2,  18 

.0048 

4.1 

.03 

No.  Confusers 

3,27 

.0009 

2.9 

NS 

Orientation 

4,36 

.0257 

6.1 

.016 

Resolution  x  Clutter 

2,  18 

.0056 

2.4 

NS 

Resolution  x  No.  Confusers 

3,27 

.0006 

2.0 

NS 

Resolution  x  Orientation 

4,  36 

.0028 

<  1 

NS 

Clutter  X  No.  Confusers 

6,  54 

.0004 

1.8 

NS 

Clutter  X  Orientation 

8,72 

.01 14 

12 

.001 

No.  Confusers  x  Orientation 

12,  108 

.0006 

3.9 

.02 

Resolution  x  Clutter  x  No.  Confusers 

6,  54 

.0007 

3.0 

NS 

Resolution  x  Clutter  x  Orientation 

8,72 

.0071 

3.3 

.03 

Resolution  x  No.  Confusers  x  Orientation 

12,  108 

.0005 

4.2 

.005 

Clutter  X  No.  Confusers  x  Orientation 

24,216 

.0005 

3.8 

.008 

Resolution  x  Clutter  x  No.  Confusers  x  Orientation 

24,216 

.0004 

2.5 

NS 

The  Clutter  x  Orientation  interaction  is  portrayed  graphically  in  FigUi'e  1 1 .  As  can  be  seen  in  the 
figure,  the  proportion  of  false  alarms  tended  to  be  higher  overall  when  the  vehicle  orientation  was  either 
0°  or  180°  as  opposed  to  45°,  90°,  or  135°.  In  addition,  the  low,  medium,  and  high  clutter  backgrounds 
exhibited  little  variation  within  each  orientation,  except  at  the  180°  level.  Post  hoc  correlated  r-tests  were 
conducted  to  test  the  statistical  significance  of  the  differences  among  the  low,  medium,  and  high  clutter 
backgrounds  within  each  orientation.  As  expected  on  the  basis  of  the  plot  in  Figure  1 1 ,  high  clutter 
differed  significantly  from  low  {p  <  .007)  and  medium  (p  <  .009)  at  the  1 80°  orientation.  Specifically, 
the  proportion  of  false  alarms  was  elevated  in  the  high  clutter  background  as  compared  to  low  and 
medium  clutter  when  the  vehicle  orientation  was  180°.  No  other  differences  were  statistically  significant. 
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Figure  11:  Mean  proportion  of  false  alarms  as  a  function  of  vehicle  orientation  for  the  low  (L), 
medium  (M),  and  high  (H)  clutter  backgrounds. 

The  interaction  between  number  of  confusers  and  orientation  is  depicted  in  Figure  12.  Inspection 
of  the  figure  reveals  that  the  proportions  of  false  alarms  associated  with  each  level  of  confusers  were 
nearly  identical  at  all  orientations  except  180°,  where  variations  among  the  four  levels  of  confusers  were 
apparent.  Namely,  the  proportion  of  fal.se  alarms  was  greatest  when  two  confusers  were  pre,sent  and 
lowest  when  no  confusers  appeared  in  the  image.  In  fact,  this  was  the  only  difference  to  reach  statistical 
significance  during  post  hoc  testing  {p  <  .002). 


Figure  12:  Mean  proportion  of  false  alarms  as  a  function  of  vehicle  orientation  for  each  number  of 
confusers. 

The  Resolution  x  Clutter  x  Orientation  interaction  appears  in  Figure  13,  which  plots  the 
proportion  of  false  alarms  as  a  function  of  vehicle  orientation  for  each  level  of  background  clutter  and 
image  resolution.  As  can  be  seen  in  the  figure,  the  interaction  appears  to  stem  primarily  from  variations 
among  the  three  levels  of  clutter  at  the  2  ft  resolution  when  the  orientation  was  1 80°.  In  that  condition. 
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the  proportion  of  false  alarms  was  noticeably  higher  in  the  high  clutter  background  as  compared  to  low 
and  medium  clutter.  The  same  relationship  was  not  observed  at  the  4  ft  resolution,  where  there  was  little 
variation  among  the  three  levels  of  clutter  at  each  vehicle  orientation.  Post  hoc  correlated  r-tests  revealed 
that  none  of  the  differences  among  the  three  levels  of  clutter  within  each  resolution  and  orientation  was 
large  enough  to  attain  statistical  significance  at  the  .01  level. 


Figure  13:  Mean  proportion  of  false  alarms  as  a  function  of  vehicle  orientation  in  the  low  (L), 
medium  (M),  and  high  (H)  clutter  backgrounds  at  the  2  ft  and  4  ft  resolutions. 

The  three-way  interaction  between  resolution,  number  of  confusers,  and  orientation  is  portrayed 
in  the  two  plots  of  Figure  14.  The  first  plot  shows  the  proportion  of  false  alarms  as  a  function  of  vehicle 
orientation  for  each  number  of  confusers  at  the  2  ft  resolution.  The  second  plot  shows  the  same 
relationship  at  the  4  ft  resolution.  At  both  resolutions,  there  tended  to  be  little  variation  among  the  four 
levels  of  confusers  at  each  orientation.  The  only  exception  was  the  180®  orientation  at  the  2  ft  resolution, 
where  the  false  alarms  were  greatest  when  two  confusers  were  present.  Post  hoc  testing  indicated  that 
the  false  alarms  were  indeed  significantly  greater  when  two  confusers  appeared  in  a  scene  than  when 
zero  ip  <  .004),  one  {p  <  .007),  or  three  ip  <  .004)  were  present,  but  only  when  the  orientation  was  180° 
at  the  2  ft  resolution.  No  other  differences  were  statistically  significant  {p  >  .01). 
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Figure  14:  Mean  proportion  of  false  alarms  as  a  function  of  vehicle  orientation  for  each  number  of 

confusers  at  the  2  ft  (left)  and  4  ft  (right)  resolutions. 

The  final  significant  interaction  in  the  analysis  of  false  alarms  was  the  Clutter  x  Number  of 
Confusers  x  Orientation  interaction,  which  appears  in  the  three  plots  of  Figure  15.  The  first  plot  shows 
the  proportion  of  false  alarms  as  a  function  of  vehicle  orientation  for  each  number  of  confusers  in  the  low 
clutter  background.  The  second  plot  shows  the  same  relationship  in  medium  clutter,  and  the  third  depicts 
high  clutter.  Examination  of  the  three  plots  reveals  that  little  performance  variation  occurred  among  the 
levels  of  confusers  within  each  level  of  orientation  in  the  low  and  medium  clutter  backgrounds. 

However,  in  the  high  clutter  background,  there  was  some  disparity  at  the  highe.st  orientation.  Namely, 
when  two  confusers  were  present,  the  false  alarms  were  higher  than  when  zero,  one,  or  three  confusers 
appeared  in  the  image.  Post  hoc  correlated  /-tests  were  conducted  to  determine  whether  any  differences 
among  the  numbers  of  confusers  within  each  orientation  and  clutter  background  were  statistically 
significant.  As  expected  from  inspection  of  Figure  15,  the  only  significant  differences  occurred  in  the 
high  clutter  background  when  the  vehicle  orientation  was  180° — the  false  alarms  were  significantly 
higher  when  two  confusers  were  present  as  compared  to  when  there  were  zero  {p  <  .007)  or  three  (p  < 
.008). 
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Figure  15:  Mean  proportion  of  false  alarms  as  a  function  of  vehicle  orientation  for  each  number  of 
confusers  in  the  low  (left),  medium  (right),  and  high  (bottom  center)  clutter  backgrounds. 

Perceptual  Sensitivity  and  Response  Bias 

The  proportions  of  hits  and  false  alarms  in  each  condition  were  used  to  derive  the  signal 
detection  theory  measures  of  perceptual  sensitivity  (d)  and  response  bias  (c).  Although  the  task  was  set 
up  as  a  detection-plus-localization  task,  we  did  not  derive  separate  estimates  of  sensitivity  and  bias  for 
detection  and  localization.  Instead,  as  described  earlier,  we  used  participants’  detection  and  localization 
responses  to  obtain  a  single  “true”  hit  rate  and  a  false  alarm  rate  for  each  image.  The  hit  and  false  alarm 
proportions  were  then  used  to  obtain  d  and  c  via  computing  formulae  3  and  4  (see  Introduction).  Prior  to 
calculating  these  indices,  proportions  of  0  and  1  were  first  adjusted  by  means  of  the  procedure 
recommended  by  Snodgrass  and  Corwin  (1988)  to  permit  the  derivation  of  perceptual  sensitivity  and  bias 
when  such  values  are  encountered. 
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Perceptual  Sensitivity. 

The  mean  perceptual  sensitivity  in  each  condition  of  resolution,  background  clutter,  number  of 
confusers,  and  vehicle  orientation  is  portrayed  in  Table  5.  The  overall  d  of  3.4  in  the  study  falls  within 
the  range  of  what  would  be  classified  as  a  moderately  easy  task,  according  to  guidelines  provided  by 
Craig  (1984).  That  is,  the  task  of  detecting  the  TEL  target  when  confusers  could  potentially  be  present 
can  be  characterized  as  moderately  easy  for  the  operators.  As  can  be  seen  in  Table  5,  the  level  of 
sensitivity  remained  stable  at  3.4  despite  fluctuations  in  image  resolution  and  the  number  of  confusers 
present  in  the  scene.  Within  each  image  resolution,  however,  there  did  appear  to  be  some  variation  as  a 
result  of  vehicle  orientation.  Specifically,  sensitivity  tended  to  be  lowest  when  orientation  was  either  0° 
or  1 80°.  A  summary  of  the  ANOVA  of  the  means  in  Table  5  appears  in  Table  6. 
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MEAN 


Table  6.  Summary  of  a  2  (resolution)  x  3  (clutter)  x  4  (no.  confusers)  x  5  (orientation)  ANOVA  of 
Perceptual  Sensitivity. 


Source 

df 

MS 

F 

P 

Resolution 

U9 

0.1 1 

<  1 

NS 

Clutter 

2,  18 

1.0 

8.6 

.003 

No.  Confusers 

3,27 

0.26 

2.4 

NS 

Orientation 

4,  36 

7.2 

7.9 

.0006 

Resolution  x  Clutter 

2,  18 

0.77 

3.7 

.04 

Resolution  x  No.  Confusers 

3,  27 

0.04 

<  1 

NS 

Resolution  x  Orientation 

4,  36 

0.70 

1.0 

NS 

Clutter  X  No.  Confusers 

6,  54 

0.08 

1.5 

NS 

Clutter  X  Orientation 

8,  72 

0.81 

5.3 

.0005 

No.  Confusers  x  Orientation 

12,  108 

0.16 

2.0 

NS 

Resolution  x  Clutter  x  No.  Confusers 

6,  54 

0.14 

1.7 

NS 

Resolution  x  Clutter  x  Orientation 

8,72 

0.61 

4.6 

.002 

Resolution  x  No.  Confusers  x  Orientation 

12,  108 

0.21 

5.2 

.0001 

Clutter  X  No.  Confusers  x  Orientation 

24,216 

0.20 

4.0 

.0001 

Resolution  x  Clutter  x  No.  Confusers  x  Orientation 

24.216 

0.20 

2.9 

.006 

As  expected  from  examination  of  the  means  in  Table  5,  the  main  effects  for  resolution  and 
number  of  confusers  were  not  statistically  significant;  however,  the  effects  for  clutter  and  vehicle 
orientation  were.  Mean  sensitivity  was  slightly  higher  in  the  low  clutter  background  {M  =  3.5,  SD  =  0.2) 
as  compared  to  medium  (M  =  3.4,  SD  =  0.2)  and  high  {M  =  3.4,  SD  =  0.2).  Only  the  difference  between 
the  low  and  high  clutter  backgrounds  was  statistically  significant  {p  <  .004).  With  respect  to  the  main 
effect  for  orientation,  perceptual  sensitivity  was  lower  at  orientations  of  0°  {M  =  3.2,  SD  =  0.4)  and  180° 
{M  =  3.2,  SD  =  0.4)  than  at  45°  (M  =  3.6,  SD  =  0.2),  90°  {M  =  3.5,  SD  =  0.2),  and  135°  (M  =  3.6,  SD  = 
0.1).  Post  hoc  correlated  /-tests  revealed  that  sensitivity  was  significantly  lower  at  orientations  of  0°  and 
180°  as  compared  to  orientations  of  45°  and  135°  (p  <  .005  for  all  four  comparisons).  No  other 
comparisons  were  statistically  significant  {p  >  .01). 

The  interaction  between  resolution  and  clutter  is  portrayed  in  Figure  16.  As  can  be  seen  in  the 
figure,  sensitivity  was  higher  at  the  2  ft  resolution  as  compared  to  4  ft  in  the  low  and  medium  clutter 
backgrounds,  but  not  in  the  high  clutter  scenes  where  the  trend  reversed.  Post  hoc  correlated  /-tests  were 
conducted  to  determine  whether  the  two  levels  of  resolution  differed  within  each  level  of  clutter.  None 
of  the  differences  in  means  was  statistically  significant  (p  >  .01). 
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figure  16:  Mean  percqptnai  sensitivity  at  the  2  ft  and  4  ft  rescdntiois  in  the  lew  (L)»  medim 
and  hig^  (H)  dntter  backgrounds. 

The  Clutter  x  Orientation  interaction  is  depicted  in  Figure  17.  As  can  be  seen  in  the  fi^ne,  tteae 
were  virtually  no  differences  among  d  scores  associated  with  each  level  of  clutter  at  each  orientation. 

The  only  exception  was  the  orientation  of  180®,  where  operator  sensitivity  in  the  high  clutter  background 
dropped  noticeably  from  that  in  the  low  and  rrtedium  clutter  backgrounds.  As  expected  from  inspection 
of  Figire  17,  the  only  significant  differences  occurred  at  the  highest  vehicle  orientation,  where  sensitivity 
was  significantly  lower  in  the  high  clutter  background  as  compared  to  low  (p  <  .009)  and  medium  clutter 
(p  <  .003).  No  other  differences  were  statistically  significant  (p  >  .01). 


Figure  17:  M^n  perceptual  ^nsltivity  as  a  fuuctioi}  of  vehicle  orientadou  in  ttie  low  (L),  inediiiin 
(M),  and  high  (H)  clutter  background. 

Hgure  18  portrays  the  significait  three-way  interaction  between  resolution,  clutter,  and 
orientation.  The  nature  of  tfiis  interaction  illustrates  the  fact  that  the  significant  Clutter  x  Orientation 
interaction  depicted  in  Figure  1 7  is  further  modified  by  the  effects  of  resolution.  Specifically,  as  can 


45 


seen  in  Figure  18,  the  drop  in  sensitivity  in  the  high  clutter  background  at  the  180°  orientation  was  a 
result  of  poorer  performance  at  the  2  ft  resolution  rather  than  the  4  ft  resolution.  Indeed,  post  hoc  testing 
indicated  that  operator  sensitivity  was  significantly  poorer  in  the  high  clutter  background  as  compared  to 
low  (p  <  .003)  and  medium  (p  <  .004)  clutter,  but  only  at  the  2  ft  resolution  for  a  vehicle  orientation  of 
180°.  No  other  differences  were  significant  (p  >  .01). 


Figure  18:  Mean  perceptual  sensitivity  as  a  function  of  vehicle  orientation  in  the  low  (L),  medium 
(M),  and  high  (H)  clutter  backgrounds  at  the  2  ft  and  4  ft  resolutions. 

The  Resolution  x  Number  of  Confusers  x  Orientation  interaction  is  illustrated  in  the  two  plots  of 
Figure  19.  The  first  plot  shows  mean  perceptual  sensitivity  as  a  function  of  vehicle  orientation  for  each 
level  of  confusers  at  the  2  ft  resolution.  The  second  plot  shows  the  same  relationship  at  the  4  ft 
resolution.  These  two  plots  reveal  that  any  variation  in  d  at  each  level  of  confusers  was  confined 
primarily  to  the  orientation  of  180°  at  the  2  ft  resolution.  In  all  other  instances,  sensitivity  tended  to 
remain  stable  at  each  orientation  despite  fluctuations  in  the  number  of  confusers.  Post  hoc  correlated  t- 
tests  were  conducted  to  determine  whether  any  fluctuations  in  d  among  the  four  levels  of  confusers 
within  each  orientation  and  resolution  were  significantly  different.  The  results  indicated  that  operator 
sensitivity  was  significantly  higher  at  an  orientation  of  180°  at  the  2  ft  resolution  when  no  confusers  were 
present  as  opposed  to  when  two  confusers  appeared  in  the  scene  {p  <  .003).  Further,  in  the  same 
orientation/resolution  condition,  sensitivity  was  inexplicably  higher  when  three  confusers  were  present  as 
opposed  to  two  ip  <  .0001).  None  of  the  remaining  comparisons  attained  statistical  significance  at  the 
.01  level. 
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2  ft  Resolution  4  ft  Resolution 


0  45  90  135  180  0  45  90  135  180 

Vehicle  Orientation  (in  degrees)  Vehicle  Orientation  (in  degrees) 


Figure  19:  Mean  perceptual  sensitivity  as  a  function  of  vehicle  orientation  for  each  number  of 
confusers  at  the  2  ft  (left)  and  4  ft  (right)  resolutions. 

The  three-way  interaction  between  clutter,  number  of  confusers,  and  orientation  is  (depicted  in  the 
three  plots  of  Figure  20.  The  first  plot  shows  mean  sensitivity  as  a  function  of  vehicle  orientation  for 
each  number  of  confusers  in  the  low  clutter  background.  The  second  plot  shows  the  same  relationship 
for  medium  clutter,  and  the  third  illustrates  high  clutter.  As  can  be  seen  in  the  plots,  variations  in 
sensitivity  among  the  four  levels  of  confusers  at  each  orientation  tended  to  occur  only  in  the  high  clutter 
background  at  1 80°.  In  that  condition,  mean  d  was  noticeably  lower  when  two  confusers  were  present  as 
compared  to  the  remaining  three  levels  of  confusers.  Post  hoc  tests  revealed  this  condition  to  be  the  only 
source  of  significant  differences  in  d.  Specifically,  in  the  high  clutter  background  and  a  vehicle 
orientation  of  180°,  sensitivity  was  significantly  lower  when  two  confusers  were  present  as  compared  to 
one  (/?  <  .001)  or  three  {p  <  .0009). 
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Figure  20:  Mean  perceptual  sensitivity  as  a  function  of  vehicle  orientation  for  each  number  of 
confusers  in  the  low  (left),  medium  (right),  and  high  (bottom  center)  clutter  backgrounds. 

Finally,  in  the  analysis  of  operator  sensitivity,  the  four-way  interaction  between  resolution, 
clutter,  number  of  confusers,  and  orientation  was  statistically  significant.  The  nature  of  the  interaction  is 
portrayed  in  the  six  plots  of  Figure  2 1 .  The  first  three  plots  show  mean  sensitivity  as  a  function  of 
vehicle  orientation  for  each  number  of  confusers  at  the  2  ft  resolution  for  low,  medium,  and  high  clutter. 
The  last  three  plots  show  the  same  relationship  at  the  4  ft  resolution  for  low,  medium,  and  high  clutter. 

As  can  be  seen  in  the  six  plots,  differences  in  operator  sensitivity  among  the  four  numbers  of  confusers  at 
each  vehicle  orientation  tended  to  be  confined  to  an  orientation  of  180°  at  the  2  ft  resolution  in  the  high 
clutter  background;  namely,  when  two  confusers  were  present,  d  was  lower  than  in  the  remaining  three 
levels  of  number  of  confusers.  Post  hoc  tests  revealed  that  operator  sensitivity  was  indeed  significantly 
lower  in  this  condition  when  two  confusers  were  present  as  opposed  to  when  zero  {p  <  .002),  one  {p  < 
.0007),  or  three  (p  <  .0004)  appeared  in  the  image.  No  other  differences  were  statistically  significant  {p 
>  .01). 


2  It  Resolution  4  ft  Resolution 

Low  Clutter  Low  Clutter 
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2  ft  Resolution  4  ft  Resolution 

Medium  Clutter  Medium  Clutter 


2  ft  Resolution  4  ft  Resolution 


Figure  21:  Mean  perceptual  sensitivity  as  a  function  of  vehicle  orientation  for  each  number  of 
confusers  at  each  image  resolution  and  clutter  level. 

Response  Bias. 

An  analysis  of  response  bias  accompanied  the  examination  of  perceptual  sensitivity  to  determine 
whether  operators  tended  to  be  conservative  or  lenient  in  their  detection  responses.  Mean  values  of 
response  bias  in  each  condition  of  resolution,  clutter,  number  of  confusers,  and  orientation  appear  in 
Table  7.  The  overall  response  bias  of  0.16  indicates  that  the  operators  were  more  or  less  neutral  in  their 
decision-making  (i.e.,  they  were  not  more  inclined  to  make  “target”  responses  nor  were  they  more  biased 
toward  “nontarget”  responses).  As  can  be  seen  in  the  table,  response  bias  in  the  various  conditions 
deviated  little  from  the  grand  mean.  In  particular,  the  means  at  the  2  ft  and  4  ft  resolution  were  identical, 
and  the  means  associated  with  each  number  of  confusers  were  very  similar.  A  summary  of  the  ANOVA 
of  the  means  in  Table  7  is  presented  in  Table  8.  None  of  the  main  effects  was  statistically  significant. 
The  only  two  significant  effects  were  the  Clutter  x  Orientation  interaction  and  the  Clutter  x  Confusers  x 
Orientation  interaction. 
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(0.18)  (0.13)  (0.16)  (0.20)  (0.15)  (0.14)  (0,13)  (0.16)  (0.16)  (0.14)  (0.13)  (0.14)  (0,15) 

0.16  0.17  0.16  0.16  0.18  0.16  0.15  0.17  0.19  0.15  0.15  0.15 

(0.16)  (0.14)  (0.17)  (0.17)  (0.17)  (0.18)  (0.14)  (0.14)  (0.17)  (0.13)  (0.12)  (0.18) 

MEAN  0.16  0.16  0.17  0.15  0.16 

_ (0.16) _ (0.18) _ (0.15) _ (0.14) _ (0.16) 


Table  8.  Summary  of  a  2  (resolution)  x  3  (clutter)  x  4  (no.  confusers)  x  5  (orientation)  ANOVA  of 
Response  Bias. 


Source 

df 

MS 

F 

P 

Resolution 

1,9 

0.003 

<1 

NS 

Clutter 

2,  18 

0.02 

<  1 

NS 

No.  Confusers 

3,27 

0.02 

1.7 

NS 

Orientation 

4,  36 

0.03 

<  1 

NS 

Resolution  x  Clutter 

2,  18 

0.02 

<  1 

NS 

Resolution  x  No.  Confusers 

3,  27 

0.001 

<  1 

NS 

Resolution  x  Orientation 

4,  36 

0.04 

<  1 

NS 

Clutter  X  No.  Confusers 

6,54 

0.01 

1.4 

NS 

Clutter  X  Orientation 

8,  72 

0.15 

5.6 

.0003 

No.  Confusers  x  Orientation 

12,  108 

0.02 

1.3 

NS 

Resolution  x  Clutter  x  No.  Confusers 

6,  54 

0.006 

<  1 

NS 

Resolution  x  Clutter  x  Orientation 

8,  72 

0.05 

1.1 

NS 

Resolution  x  No.  Confusers  x  Orientation 

12, 108 

0.02 

2.3 

NS 

Clutter  X  No.  Confusers  x  Orientation 

24, 216 

0.02 

3.6 

.0001 

Resolution  x  Clutter  x  No.  Confusers  x 
Orientation 

24,216 

0.02 

2.0 

NS 

The  Clutter  x  Orientation  interaction  is  portrayed  in  Figure  22.  As  can  be  seen  in  the  figure, 
there  was  little  variation  in  response  bias  among  the  three  levels  of  clutter  within  each  orientation  when 
the  orientation  was  45°,  90°,  or  1 35°.  However,  when  the  orientation  was  0°,  response  bias  tended  to  be 
more  conservative  in  the  high  clutter  background  than  in  the  low  and  medium  clutter  scenes.  When  the 
orientation  was  180°,  this  trend  reversed  so  that  respon.se  bias  became  relatively  more  lenient  in  high 
clutter  than  in  low  and  medium  clutter.  Post  hoc  testing  revealed  that  the  only  significant  difference  was 
between  the  medium  and  high  clutter  backgrounds  at  the  0°  orientation  {p  <  .01). 


Figure  22:  Mean  response  bias  as  a  function  of  vehicle  orientation  in  the  low  (L),  medium  (M),  and 
high  (H)  clutter  backgrounds. 
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The  Clutter  x  Number  of  Confusers  x  Orientation  interaction  is  depicted  in  the  three  plots  of 
Figure  23.  The  first  plot  shows  mean  response  bias  as  a  function  of  vehicle  orientation  for  each  number 
of  confusers  in  the  low  clutter  background.  The  remaining  two  plots  show  the  same  relationship  in  the 
medium  and  high  clutter  backgrounds.  As  can  be  seen  in  the  figure,  there  was  some  variation  in  response 
bias  among  the  four  levels  of  confusers  within  each  orientation  at  all  three  clutter  levels.  The  differences 
were  most  apparent  in  the  medium  (at  the  0°  and  180°  orientations)  and  high  (at  the  180°  orientation) 
clutter  backgrounds.  Post  hoc  correlated  Mests  were  conducted  to  determine  whether  the  four  levels  of 
confusers  within  each  orientation  and  clutter  background  differed  from  one  another.  Only  the  difference 
in  response  bias  associated  with  two  versus  three  confusers  at  180°  in  the  high  clutter  background  was 
statistically  significant  (/?  <  .01).  Response  bias  was  relatively  more  conservative  when  two  confusers 
were  present. 


Low  Clunor  Modium  Cluttor 


Figure  23:  Mean  response  bias  as  a  function  of  vehicle  orientation  for  each  number  of  confusers  in 
the  low  (left),  medium  (right),  and  high  (bottom  center)  clutter  backgrounds. 


52 


Reaction  Times 


In  addition  to  examining  performance  measures  such  as  hits,  false  alarms,  sensitivity,  and  bias, 
we  also  analyzed  operators’  decision-making  reaction  times.  Reaction  times  were  classified  as  RTs  for 
hits,  correct  rejections,  misses,  or  false  alarms,  depending  on  the  image  type  and  the  operator’s  decision. 
For  example,  if  the  operator  responded  “target”  to  an  image  containing  only  the  confuser  vehicle,  the  RT 
for  that  response  was  classified  as  a  false  alarm  reaction  time. 

RT  for  Hits. 

Mean  reaction  times  for  hits  in  each  condition  of  resolution,  clutter,  number  of  confusers,  and 
orientation  are  presented  in  Table  9.  Overall,  operators  took  just  over  3  seconds  to  respond  correctly  to 
target  imagery.  As  can  be  seen  in  the  table,  they  were  somewhat  faster  at  the  2  ft  resolution  as  compared 
to  4  ft.  In  addition,  RT  tended  to  increase  progressively  as  the  number  of  confusers  increased.  Within 
each  resolution,  there  was  also  a  noticeable  tendency  for  RT  to  be  higher  at  the  0°  and  1 80°  orientations 
as  compared  to  45°,  90°,  and  135°.  A  summary  of  the  ANOVA  of  the  means  in  Table  9  appears  in  Table 
10.  As  can  be  seen  in  the  table,  all  of  the  main  effects  were  statistically  significant  as  were  a  number  of 
the  interactions. 
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Table  10.  Summary  of  a  2  (resolution)  x  3  (clutter)  x  4  (no.  confusers)  x  5  (orientation)  ANOVA  of 
Reaction  Time  for  Hits 


Source 

df 

MS 

F 

p 

Resolution 

1,9 

Ain 

5.4 

.04 

Clutter 

2,  18 

13.4 

44.3 

.0001 

No.  Confusers 

3,27 

19.1 

23.0 

.0001 

Orientation 

4,  36 

81.2 

27.5 

.0001 

Resolution  x  Clutter 

2,  18 

0.16 

<  I 

NS 

Resolution  x  No.  Confusers 

3,27 

0.5 

1.1 

NS 

Resolution  x  Orientation 

4,36 

4.9 

4.0 

.04 

Clutter  X  No.  Confusers 

6,  54 

0.2 

<  I 

NS 

Clutter  X  Orientation 

8,72 

3.0 

18.1 

.0001 

No.  Confusers  x  Orientation 

12,  108 

0.2 

1.2 

NS 

Resolution  x  Clutter  x  No.  Confusers 

6,  54 

0.3 

1.3 

NS 

Resolution  x  Clutter  x  Orientation 

8,  72 

0.3 

1.7 

NS 

Resolution  x  No.  Confusers  x  Orientation 

12,  108 

1.4 

4.2 

.003 

Clutter  X  No.  Confusers  x  Orientation 

24,216 

1.0 

4.0 

.0001 

Resolution  x  Clutter  x  No.  Confusers  x  Orientation 

24,216 

1.0 

4.2 

.0001 

With  respect  to  the  main  effect  for  resolution,  operators  were  indeed  significantly  faster  at  the  2 
ft  resolution  than  at  4  ft.  Regarding  the  main  effect  for  clutter,  RT  tended  to  be  faster  in  the  low  {M  = 

3.0,  SD  =  0.9)  and  medium  (M  =  3. 1 ,  SD  =  0.9)  clutter  backgrounds  as  compared  to  high  clutter  (M  =  3.4, 
SD  =  1 .0).  Post  hoc  testing  revealed  that  both  of  these  backgrounds  differed  significantly  from  high 
clutter  (p  <  .0001)  but  not  from  each  other  (/?  >  .01).  As  mentioned  earlier,  the  main  effect  for  number  of 
confusers  stemmed  from  the  tendency  for  RT  to  increase  with  the  number  of  confusers  present  in  the 
image.  Post  hoc  tests  indicated  that  all  of  the  confusers  differed  significantly  from  one  another  at  p  <  .01 
except  zero  and  one.  Finally,  with  respect  to  the  main  effect  for  orientation,  RT  was  faster  at  orientations 
of  45°  (M  =  2.8,  SD  =  0.9),  90°  (M  =  2.7,  SD  =  0.8),  and  135°  (M  =  2.7,  SD  =  0.9)  as  compared  to  0°  (M  = 
3.9,  SD=  1.2)  and  180°  (A/ =  3.6,  SZ)  =  1.0).  Post  hoc  tests  revealed  that  all  of  the  orientations  differed 
significantly  at  p  <  .01  except  0°  and  1 80°,  45°  and  90°,  and  90°  and  1 35°. 

The  Resolution  x  Orientation  interaction  is  portrayed  in  Figure  24.  As  can  be  seen  in  the  figure, 
the  RT  for  hits  at  the  two  image  resolutions  within  each  vehicle  orientation  differed  noticeably  from  each 
other  only  when  the  orientation  was  0°.  Post  hoc  correlated  r-tests  were  conducted  to  determine  whether 
the  two  resolutions  within  each  orientation  differed  significantly  from  each  other.  None  of  the 
differences  in  means  attained  significance  at  the  .01  level. 
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Figure  24:  Mean  RT  for  hits  as  a  function  of  vehicle  orientation  at  the  2  ft  and  4  ft  resolutions. 

The  nature  of  the  Clutter  x  Orientation  interaction  can  be  seen  in  Figure  25.  As  with  resolution, 
the  means  for  the  three  levels  of  clutter  within  each  vehicle  orientation  differed  most  noticeably  from  one 
another  at  the  0°  orientation.  Post  hoc  testing  revealed  that  all  three  clutter  backgrounds  did  indeed  differ 
significantly  from  one  another  at  the  0°  orientation  {p  <  .002).  The  RT  for  hits  was  highest  in  the  high 
clutter  background  and  improved  progressively  as  the  clutter  decreased.  Post  hoc  testing  further  revealed 
that  the  low  clutter  background  differed  from  medium  (p  <  .006)  and  high  (p  <  .0004)  at  the  135° 
orientation  and  that  the  low  and  high  backgrounds  differed  when  the  orientation  was  180°  (p  <  .009).  No 
other  differences  were  statistically  significant  (p  >  .01). 


Figure  25;  Mean  RT  for  hits  as  a  function  of  vehicle  orientation  in  the  low  (L),  medium  (M),  and 
high  (H)  clutter  backgrounds. 
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The  three-way  interaction  between  resolution,  number  of  confusers,  and  orientation  is  depicted  in 
the  two  plots  in  Figure  26.  The  first  plot  shows  mean  RT  for  hits  as  a  function  of  vehicle  orientation  for 
each  number  of  confusers  at  the  2  ft  resolution.  The  second  plot  shows  the  same  relationship  at  the  4  ft 
resolution.  Inspection  of  the  two  plots  in  Figure  26  reveals  some  variation  in  RT  for  hits  among  the  four 
numbers  of  confusers  within  each  orientation  at  both  the  2  ft  and  4  ft  resolutions.  In  particular,  the  RT 
was  obviously  higher  when  three  confusers  were  present  as  compared  to  zero  at  each  orientation. 
Follow-up  testing  of  the  means  within  each  orientation  and  resolution  produced  the  following  results. 
With  respect  to  the  2  ft  resolution,  the  RT  for  three  confusers  was  higher  than  the  RT  for  zero,  one,  or 
two  confusers  at  the  0°  orientation  {p  <  .0004).  At  45°,  only  zero  and  three  confusers  differed  from  each 
other  ip  <  .005).  At  90°,  three  confusers  differed  significantly  from  zero  {p  <  .0007),  one  {p  <  .002),  and 
two  {p  <  .004)  confusers.  At  135°,  zero  confusers  differed  from  two  and  three  {p  <  .007)  and  one 
confuser  differed  from  three  {p  <  .002).  At  180°,  zero  confusers  differed  significantly  from  one,  two,  and 
three  {p  <  .004)  as  did  one  versus  three  confusers  (/;  <  .001).  With  respect  to  the  4  ft  resolution,  none  of 
the  differences  was  statistically  significant  at  orientations  of  0°  and  180°.  At  45°  and  90°,  zero  confusers 
differed  from  two  and  three  {p  <  .006).  Finally,  at  135°,  zero  confusers  differed  from  two  and  three  {p  < 
.007)  as  did  one  versus  three  confusers  {p  <  .003). 


Figure  26:  Mean  RT  for  hits  as  a  function  of  vehicje  orientation  for  each  number  of  confusers  at 
the  2  ft  (left)  and  4  ft  (right)  resolutions. 

The  Clutter  x  Number  of  Confusers  x  Orientation  interaction  is  portrayed  in  the  three  plots  of 
Figure  27.  The  first  plot  shows  mean  RT  for  hits  as  a  function  of  vehicle  orientation  for  the  four 
numbers  of  confusers  in  the  low  clutter  background.  The  second  plot  shows  the  same  relationship  in  the 
medium  clutter  background,  and  the  third  plot  depicts  the  high  clutter  background.  In  the  low  clutter 
background,  RT  for  hits  appeared  to  be  higher  for  two  and  three  confusers  as  compared  to  zero  and  one  at 
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an  orientation  of  0°.  In  addition,  at  180°,  the  RT  associated  with  three  confusers  was  noticeably  higher 
than  the  RT  for  all  other  levels  of  confusers.  In  the  medium  clutter  scenes,  there  was  a  clear  discrepancy 
between  zero  and  three  confusers  within  all  vehicle  orientations,  though  this  difference  was  smallest  at 
0°.  Finally,  in  the  high  clutter  background,  the  least  variation  among  the  levels  of  confusers  occurred  at 
45°.  At  0°,  90°,  and  135°,  RT  associated  with  three  confusers  tended  to  be  elevated;  at  180°,  it  was  the 
RT  for  two  confusers  that  stood  out.  The  entries  in  Table  1 1  show  which  of  these  ob.servations  were 
statistically  significant  during  post  hoc  testing. 


Low  Clutter  Medium  Clutter 


Figure  27:  Mean  RT  for  hits  as  a  function  of  vehicle  orientation  for  each  number  of  confusers  in 
the  low  (left),  medium  (right),  and  high  (bottom  center)  clutter  backgrounds. 
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Table  11.  Results  of  Post  Hoc  Tests  for  the  Clutter  x  Number  of  Confusers  x  Orientation 
Interaction  in  the  Analysis  of  RT  for  Hits. 
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Finally,  the  ANOVA  of  RT  for  hits  revealed  a  significant  four-way  interaction,  which  is  depicted 
in  the  six  plots  of  Figure  28.  The  first  three  plots  show  mean  RT  for  hits  as  a  function  of  vehicle 
orientation  for  the  four  numbers  of  confusers  in  low,  medium,  and  high  clutter  backgrounds,  respectively, 
at  the  2  ft  resolution.  The  last  three  plots  show  the  same  relationships  in  the  low,  medium,  and  high 
clutter  backgrounds  at  the  4  ft  resolution.  As  can  be  seen  in  the  figure,  the  interaction  stems  from  the  fact 
that  the  variation  among  the  four  levels  of  confusers  within  each  orientation  differed  at  each  resolution 
and  background.  For  example,  in  the  low  clutter  background  at  the  2  ft  resolution,  there  was  a  noticeable 
discrepancy  between  the  RT  for  three  confusers  and  the  RT  associated  with  zero,  one,  and  two  confusers 
at  an  orientation  of  180°.  This  relationship  was  not  evident  in  the  remaining  five  plots.  The  results  of 
post  hoc  testing  are  summarized  in  Table  12  (2  ft  resolution)  and  Table  13  (4  ft  resolution).  Inspection  of 
the  two  tables  reveals  that  23  of  the  90  comparisons  were  statistically  significant  at  the  2  ft  resolution, 
whereas  only  14  reached  significance  at  the  4  ft  resolution.  Further,  at  the  4  ft  resolution,  all  of  the 
significant  differences  but  one  occurred  at  orientations  of  45°,  90°,  and  135°.  At  the  2  ft  resolution,  on 
the  other  hand,  the  majority  of  the  significant  differences  occurred  at  orientations  of  0°  and  180°. 
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Table  12.  Results  of  Post  Hoc  Tests  for  the  Resolution  x  Clutter  x  Number  of  Confusers  x 
Orientation  Interaction  in  the  Analysis  of  RT  for  Hits  (2  ft  Resolution). 
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Table  13.  Results  of  Post  Hoc  Tests  for  the  Resolution  x  Clutter  x  Number  of  Confusers  x 
Orientation  Interaction  in  the  Analysis  of  RT  for  Hits  (4  ft  Resolution). 
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RT  for  Correct  Rejections. 

Mean  reaction  times  for  correct  rejections  in  each  condition  of  resolution,  clutter,  number  of 
confusers,  and  orientation  are  presented  in  Table  14.  The  overall  mean  of  4.1  seconds  Indicates  that  the 
operators  took  slightly  longer  to  make  a  correct  response  to  nontargets  than  to  targets  (where  the  mean 
was  3.2  seconds).  The  RT  for  correct  rejections  was  noticeably  higher  at  the  4  ft  resolution  as  compared 
to  the  2  ft  resolution.  There  was  a  slight  tendency  for  RT  to  increase  as  the  number  of  confusers 
increased.  Within  each  resolution,  the  RT  associated  with  orientations  of  0°  and  1 80°  tended  to  be  higher 
than  the  RT  for  the  remaining  three  orientations.  A  summary  of  the  ANOVA  of  the  means  in  Table  14 
appears  in  Table  15.  All  four  of  the  main  effects  were  statistically  significant. 
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Table  14.  Mean  Reaction  Time  (seconds)  for  Correct  Rejections  (Standard  Deviations  in  Parentheses)  in  Each  Condition. 


C/3 

Di 

LU  CN 

oo 

D 

Uh 

Z 

o 

u 

u. 

c 

tc 

LU 

CQ 

S 

D 

Z 


D::: 

m  _ 

D 

U 
Q 
Z 
D 
O 

a 

u  x 
< 
ca 


Tt  —  *ro— 


fovqcNOcvjcNOvqr^ON 


tt— —  CO  —  T^^  — 


ootoOooor'OoqascNi 


vqp*>csvqc?\(N»/3r^vq'^ 


r-  Cvl  ;  00  P-;  XT  \q  — ; 

ro  —  CO  — *  rS  O 


OCNOnO  —  OOOOOCNO 
rn— •(NOr^OrnOro  — 


o  On  —  i/^i  O  tn 

^  T^‘  _  ^  - -  Tf  « 


r^!  CO  O  — 


ro  —  or^oor^CNoo^o 

CO— ^roOcNOcNOro  — 


csi  ‘  CO  — ‘  evi 


oocor40cNO^triroTrO 

co^ro  —  cooco— ^co— ^ 


osvor^r^ONoooNO 

CNOCNOCNOCN^ 


CO  ^ 


vq  \q 

T^r  *— 


r-;  CN 
CO  — 


CO  — 


to  oo 

— 


XT)  — 
erj  ^ 


CN  O 

CO  O 


— 


p  O) 
CO  O 


TJ;  r-; 
TJ-*  — ^ 


CO 


0\  oo 
CN  O 


to 

tT 


^  to  o 

§  :2  22 


c 

e3 

o 


OOppCNlO— ^totooo 
tOCN’^---Tf----Tf---Tt*-^ 


p^(NCN*^r-ppTf^ 

to  CN  to  CN  to  — «*  to  — ^  to  CN 


pr-covCcNtoo^rTt'O 
rg-*  ^  ^  ^ 


Or-;0CpP-*— ^r^CNCNtO 

^  fr^  ^  fT']  ^  («ri  ^  t;j-'  « 


CNtOT^TfOpCvOO  —  CN 
^  CN  to  CN  to  — ^  TT  —  to  CN 


ppCNpppp'^CNp 
to  CN  Tj*  Tt  CO  — -  Tt  — 


CNr-^pppCNp-~;pTr 


pCO  —  CNr-OCC^OO  —  CO 
tOCNtOCN^r  —  TT'—tooi 


pr-;OOpor^cocNr^ 


— -totocNpcNrr— 
TT  —  CO— ’co— *ro  — 


r^cNpcNoooooooorr^T 
tooitocNxr— -Tf-^tocN 


OpOOptOppCNtO 


^co  —  ro^co— ^ro^ 


T^r  — 


p 

to  — ^ 


CO  to 


q 

— 


p  p 
^  — 


O  CO 


rs|  — . 
to  CN 


—  p 

T3-*  — 


OO  p 
cr,  ^ 


to  CN 


«  p 
TT  — - 


p  CN 
CO 


to 

Tj- 


b  ^ 

§  22  22  ;! 


—  p 


p  oo 
to  ^ 


p  TT  CN  p 
^  — 


p  p 
CO  — ‘ 


p  q 

TT  CN 


p  —  P 

CO  — 


p  p 
— 


oo  O  VO 

CO  —  Tt 


oq  p 


p  ra;  p  p 
CO  TT  — * 


62 


MEAN 


Table  15.  Summary  of  a  2  (resolution)  x  3  (clutter)  x  4  (no.  confusers)  x  5  (orientation)  ANOVA  of 
Reaction  Time  for  Correct  Rejections. 
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3.7 
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0.3 
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With  respect  to  the  main  effect  for  resolution,  the  RT  for  correct  rejections  was  indeed 
significantly  elevated  at  the  4  ft  resolution  as  compared  to  2  ft.  The  main  effect  for  clutter  arose  from  the 
progressive  increase  in  RT  as  the  clutter  increased  from  low  (M  =  3.5,  SD  =  1 .0)  to  medium  {M  =  3.9,  SD 
=  1 .3)  to  high  (M  =  4.9,  SD  =  1 .8).  Post  hoc  correlated  Mests  indicated  that  the  RT  for  correct  rejections 
was  significantly  higher  in  the  high  clutter  background  as  compared  to  low  (p  <  .0012)  and  medium  (p  < 
.0004).  As  was  observed  in  Table  14,  the  main  effect  for  confusers  was  a  result  of  the  slight  tendency  for 
RT  to  increase  as  the  number  of  confusers  in  the  scene  increased.  Follow-up  testing  revealed  that  the  RT 
for  zero  or  one  confusers  was  significantly  faster  than  for  two  or  three  confusers  (p  <  .004  for  all  four 
comparisons).  No  other  comparisons  were  statistically  significant  (/;  >  .01).  Finally,  with  respect  to  the 
main  effect  for  vehicle  orientation,  the  RT  was  elevated  at  0°  (M  =  4.5,  SD  =  1 .6)  and  1 80°  (M  =  4.2,  SD 
=  1 .5)  as  compared  to  45°  (M  =  3.9,  SD  =  1 .3),  90°  (M  =  3.8,  SD  =  1 .2),  and'  1 35°  {M  =  3.9,  SD=\  .3). 

Post  hoc  testing  indicated  that  the  RT  at  0°  was  significantly  higher  when  compared  to  45°  (p  <  .001),  90° 
(p  <  .006),  and  135°  (p  <  .003).  No  other  differences  were  statistically  significant  (p  >  .01). 


The  Resolution  x  Orientation  interaction  is  portrayed  graphically  in  Figure  29.  As  can  be  seen  in 
the  figure,  mean  RT  for  correct  rejections  at  the  4  ft  resolution  was  elevated  in  comparison  to  the  2  ft 
resolution,  though  the  difference  tended  to  decrease  as  the  vehicle  orientation  increased  from  0°  to  180°. 
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The  results  of  post  hoc  correlated  r-tests  indicated  that  the  two  resolutions  differed  (p  <  .004)  at  all 
orientations  except  180°. 


Figure  29:  Mean  RT  for  correct  rejections  as  a  function  of  vehicle  orientation  at  the  2  ft  and  4  ft 
resolutions. 

The  Resolution  x  Confusers  x  Orientation  interaction  is  depicted  in  the  two  plots  of  Figure  30. 
The  first  plot  shows  mean  RT  for  correct  rejections  as  a  function  of  vehicle  orientation  for  each  number 
of  confusers  at  the  2  ft  resolution.  The  second  plot  shows  the  same  relationship  at  the  4  ft  resolution. 
Inspection  of  the  two  plots  reveals  that  there  was  very  little  variation  among  the  four  numbers  of 
confusers  within  each  orientation  at  the  4  ft  resolution.  However,  at  the  2  ft  resolution,  there  were  some 
noticeable  differences  at  0°  and  180°.  Follow-up  testing  indicated  that  at  the  2  ft  resolution,  the  RT  was 
significantly  higher  for  three  confusers  as  compared  to  zero,  one,  and  two  at  orientations  of  0°  (/?  <  .0008) 
and  90°  {p  <  .003).  At  45°,  only  zero  and  three  confusers  differed  {p  <  .0043).  At  both  135°  and  at  180°, 
zero  confusers  differed  from  two  and  three  (p  <  .007),  and  one  confuser  differed  from  three  (p  <  .009). 
Turning  to  the  4  ft  resolution,  none  of  the  differences  was  statistically  significant  at  either  0°  or  180°.  At 
45°,  zero  confusers  differed  from  two  and  three  (p  <  .004),  and  one  differed  from  two  (p  <  .008).  At  90°, 
zero  confusers  differed  from  two  and  three  (p  <  .004).  Finally,  at  135°,  zero  confusers  differed  from  two 
(p  <  .01)  and  three  (p  <  .001),  and  one  confuser  differed  from  three  (p  <  .002). 
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2  ft  Resolution 


4  ft  Resolution 


Vehicle  Orientation  (in  degrees)  O^emation  (in  degrees) 


Figure  30:  Mean  RT  for  correct  rejections  as  a  function  of  vehicle  orientation  for  each  number  of 
confusers  at  the  2  ft  (left)  and  4  ft  (right)  resolutions. 

The  interaction  between  clutter,  number  of  confusers,  and  orientation  is  portrayed  in  the  three 
plots  of  Figure  3 1 .  The  first  plot  shows  mean  RT  for  correct  rejections  as  a  function  of  vehicle 
orientation  for  each  number  of  confusers  in  the  low  clutter  background.  The  second  plot  shows  the  same 
relationship  in  the  medium  clutter  background,  and  the  third  plot  depicts  high  clutter.  Inspection  of  the 
three  plots  indicates  that  the  RTs  for  the  four  numbers  of  confusers  within  each  orientation  were  most 
similar  in  the  medium  clutter  background.  In  low  clutter,  the  RT  for  three  confusers  was  noticeably 
elevated  at  0°  and  at  180°.  In  high  clutter,  the  RT  for  two  confusers  was  inexplicably  higher  at  180°.  A 
summary  of  the  results  of  post  hoc  testing  for  RT  for  correct  rejections  is  presented  in  Table  16.  The 
entries  in  the  table  reveal  that  20  of  the  90  comparisons  were  statistically  significant.  Of  those,  14 
represented  significant  differences  between  zero,  one,  or  two  confusers  versus  three  confusers,  implying 
that  increases  in  RT  for  correct  rejections  will  become  most  widespread  when  the  number  of  confusers 
increases  to  three. 


Low  Clutter  Medium  Clutter 
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6.00 


High  Clutter 


Figure  31:  Mean  RT  for  correct  rejections  as  a  function  of  vehicle  orientation  for  each  number  of 
confusers  in  the  low  (left),  medium  (right),  and  high  (bottom)  clutter  backgrounds. 


Table  16.  Results  of  Post  Hoc  Tests  for  the  Clutter  x  Number  of  Confusers  x  Orientation 
Interaction  in  the  Analysis  of  RT  for  Correct  Rejections. 
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Finally,  the  ANOVA  of  RT  for  correct  rejections  revealed  a  significant  four-way  interaction 
between  resolution,  clutter,  number  of  confusers,  and  orientation.  The  nature  of  this  interaction  can  be 
seen  in  the  six  plots  of  Figure  32.  The  first  three  plots  show  mean  RT  for  correct  rejections  as  a  function 
of  vehicle  orientation  for  the  four  numbers  of  confusers  in  the  low,  medium,  and  high  clutter 
backgrounds  at  the  2  ft  resolution.  The  remaining  three  plots  show  the  same  relationships  in  the  low, 
medium,  and  high  clutter  backgrounds  at  the  4  ft  resolution.  A  summary  of  the  results  of  post  hoc  tests 
for  this  interaction  appear  in  Table  17  (2  ft  resolution)  and  Table  18  (4  ft  resolution).  As  with  the 
significant  four-way  interaction  in  the  analysis  of  RT  for  hits,  most  of  the  significant  comparisons 
occurred  at  the  2  ft  resolution  (23  of  90)  as  opposed  to  4  ft  (only  1 3  of  90).  Further,  at  the  4  ft  resolution, 
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there  were  no  significant  differences  at  orientations  of  0°  and  180°.  By  comparison,  at  the  2  ft  resolution 
the  significant  differences  spanned  all  levels  of  vehicle  orientation. 


2  ft  Resolution 
Low  Clutter 


4  ft  Resolution 
Low  Clutter 


Vehicle  Orientation  (in  degrees) 


2  ft  Resolution 
Medium  Clutter 


Vehicle  Orientation  (in  degrees) 


4  ft  Resolution 
Medium  Clutter 


2  ft  Resolution 


Vehicle  Orientation  (in  degrees) 


4  ft  Resolution 


Figure  32:  Mean  RT  for  hits  as  a  function  of  vehicle  orientation  for  each  number  of  confusers  at 
each  image  resolution  and  clutter  level. 
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Table  17.  Results  of  Post  Hoc  Tests  for  the  Resolution  x  Clutter  x  Number  of  Confusers  x 
Orientation  Interaction  in  the  Analysis  of  RT  for  Correct  Rejections  (2  ft  Resolution). 
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Table  18.  Results  of  Post  Hoc  Tests  for  the  Resolution  x  Clutter  x  Number  of  Confusers  x 
Orientation  Interaction  in  the  Analysis  of  RT  for  Correct  Rejections  (4  ft  Resolution). 
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RT  for  Misses  and  False  Alarms. 

Because  most  of  the  responses  made  by  the  operators  in  this  study  tended  to  be  correct  (i.e., 
either  hits  or  correct  rejections),  there  were  much  fewer  incorrect  responses  (i.e.,  misses  and  false  alarms) 
available  for  analysis.  In  fact,  some  operators  did  not  make  any  mistakes  in  some  of  the  conditions, 
which  meant  that  RT  data  for  misses  and  false  alarms  were  missing  entirely.  Consequently,  because  of 
the  presence  of  missing  data  in  some  conditions,  full  factorial  analyses  of  RT  for  misses  and  RT  for  false 
alarms  could  not  be  completed.  Instead,  only  main  effects  analyses  were  conducted  for  each  of  the  four 
independent  variables.  The  results  of  these  analyses  are  presented  here. 
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First,  none  of  the  main  effects  for  resolution,  clutter,  number  of  confusers,  or  orientation  was 
statistically  significant  (p  >  .05)  in  the  analyses  of  RT  for  false  alarms.  Second,  in  the  analyses  of  RT  for 
misses  (N  =  9),  only  the  main  effects  for  clutter  and  number  of  confusers  were  statistically  significant. 
With  respect  to  the  main  effect  for  clutter,  F  (2,16)  =  5.6,  p  <  .015,  the  RT  for  misses  was  much  higher  in 
the  high  clutter  background  (M  =  8. 1 ,  SD  =  3.6)  as  compared  to  the  low  (M  =  5.9,  SD  =  3.0)  and  medium 
{M  =  5.6,  SD  =  2.3)  clutter  backgrounds.  Post  hoc  correlated  t-tests  indicated  that  the  discrepancies 
between  the  high  versus  the  low  and  medium  clutter  scenes  were  statistically  significant,  p  <  .01 . 
Regarding  the  main  effect  for  number  of  confusers,  F  (3,  24)  =  3.6,  p  <  .03,  the  RT  for  misses  was  much 
higher  when  three  confusers  were  present  (M  =  7.8,  SD  =  4.2)  as  compared  to  zero  (M  =  6.4,  SD  -  2.6), 
one  (M  =  5.7,  SD  =  3.0),  and  two  (M  =  6.4,  SD  -  2.8).  However,  because  of  the  high  standard  deviations 
associated  with  the  means,  none  of  the  differences  was  statistically  significant,  p  >  .01 . 

Comparison  of  RT  for  Hits.  Correct  Reiections.  Misses,  and  False  Alarms. 

The  final  analysis  of  operators’  reaction  times  constituted  a  comparison  of  their  RTs  for  hits, 

correct  rejections,  misses,  and  false  alarms.  In  general,  a  rank  ordering  of  the  RTs  typically  shows  that 
operators  are  faster  when  making  correct  decisions  (hits  and  correct  rejections)  as  compared  to  incorrect 
decisions  (misses  and  false  alarms;  See  &  Kuperman,  1995).  Figure  33  portrays  mean  reaction  times  for 
hits,  correct  rejections,  misses,  and  false  alarms  in  the  present  investigation.  As  can  be  seen  in  the  figure, 
reaction  times  for  correct  decisions  were  indeed  faster  than  those  for  incorrect  decisions  in  this  study. 
Further,  RT  for  hits  tended  to  be  faster  than  RT  for  correct  rejections  and  RT  for  misses  tended  to  be 
faster  than  RT  for  false  alarms.  A  one-way  ANOVA  of  the  RTs  was  statistically  significant,  F  (3,24)  = 
10. 1,  p  <  .004.  Follow-up  tests  indicated  that  all  of  the  RTs  were  significantly  different  from  one  another 
at  p  <  .01  except  for  misses  and  false  alarms. 


Response  Category 


Figure  33:  Mean  RT  for  hits,  correct  rejections,  misses,  and  false  alarms. 
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Operator  Confidence  Ratings 


Although  operators  were  asked  to  rate  their  confidence  in  each  target  and  nontarget  decision 
primarily  to  provide  the  data  needed  to  generate  ROC  curves,  we  decided  to  conduct  a  separate  analysis 
of  their  ratings  to  examine  the  effects  of  each  independent  variable  on  decision  confidence  alone.  When 
deciding  whether  or  not  a  TEL  was  present,  operators  were  asked  to  respond  1, 2,  or  3  if  they  thought  a 
target  was  definitely,  probably,  or  possibly  not  present  and  4,  5,  or  6  if  they  thought  the  TEL  was 
possibly,  probably,  or  definitely  present.  Hence,  their  responses  represented  low,  medium,  and  high 
levels  of  confidence  in  each  target  and  nontarget  decision.  The  target/nontarget  confidence  ratings  were 
converted  to  a  scale  ranging  from  one  (low  confidence)  to  three  (high  confidence)  for  subsequent 
analysis. 

Mean  confidence  ratings  in  each  condition  of  resolution,  clutter,  number  of  ccnfusers,  and 
orientation  are  presented  in  Table  19.  The  overall  mean  of  2.32  indicates  that  the  operators  were 
somewhat  more  than  moderately  confident  in  their  responses.  Further,  the  means  in  the  table  reveal  that 
they  were  more  confident  when  viewing  the  2  ft  resolution  imagery  as  compared  to  4  ft.  However,  their 
confidence  did  not  appear  to  be  affected  by  an  increase  in  the  number  of  confusers  present  in  the  scene. 
Within  each  level  of  image  resolution,  there  was  a  noticeable  tendency  for  the  mean  confidence  rating  to 
be  higher  at  vehicle  orientations  of  45^  90^  and  135"^  as  opposed  to  0°  and  180°.  A  summary  of  the 
ANOVA  of  the  means  in  Table  19  appears  in  Table  20.  All  of  the  main  effects  except  number  of 
confusers  were  statistically  significant. 
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2.32  2.28  2.27  2.45  2.37  2.22  2.35  2.35  2.27  2.35  2.35  2.32 

(0.70)  (0.71)  (0.73)  (0.47)  (0.49)  (0.54)  (0.68)  (0.68)  (0.70)  (0.67)  (0.66)  (0.68) 

MEAN  2.29  2.35  2.32  2.34  2.32 

_ (0.71)  _ (0.51) _ (0.69) _ (0.67) _ (0.65) 


Table  20.  Summary  of  a  2  (resolution)  x  3  (clutter)  x  4  (no.  confusers)  x  5  (orientation)  ANOVA  of 
Operator  Confidence  Ratings. 


Source 

df 

MS 

F 

P 

Resolution 

1,9 

8.2 

17.5 

.002 

Clutter 

2,  18 

0.97 

15.8 

.001 

No.  Confusers 

3,  27 

0.20 

1.1 

NS 

Orientation 

4,36 

52.5 

36.5 

.0001 

Resolution  x  Clutter 

2,  18 

0.02 

3.3 

NS 

Resolution  x  No.  Confusers 

3,  27 

0.03 

<1 

NS 

Resolution  x  Orientation 

4,36 

1.6 

9.6 

.003 

Clutter  X  No.  Confusers 

6,  54 

0.22 

6.0 

.006 

Clutter  X  Orientation 

8,  72 

0.02 

1.5 

NS 

No.  Confusers  x  Orientation 

12,  108 

0.78 

18.3 

.0001 

Resolution  x  Clutter  x  No.  Confusers 

6,  54 

0.01 

<1 

NS 

Resolution  x  Clutter  x  Orientation 

8,  72 

0.01 

1.5 

NS 

Resolution  x  No.  Confusers  x  Orientation 

12,  108 

0.07 

4.2 

.0006 

Clutter  X  No.  Confusers  x  Orientation 

24,216 

0.02 

1.6 

NS 

Resolution  x  Clutter  x  No.  Confusers  x 
Orientation 

24,216 

0.01 

1.2 

NS 

With  respect  to  the  main  effect  for  resolution,  operator  confidence  was  indeed  significantly 
higher  at  the  2  ft  resolution  as  compared  to  4  ft.  The  main  effect  for  background  clutter  revealed  that 
confidence  tended  to  be  higher  in  low  {M  =  2.37,  SD  =  0.40)  and  medium  clutter  (M  =  2.34,  SD  =  0.40) 
as  compared  to  high  (M  =  2.27,  SD  =  0.43).  The  results  of  post-hoc  correlated  Mests  indicated  that  both 
low  and  medium  clutter  differed  significantly  from  high  clutter,  p  <  .003.  Finally,  the  main  effect  for 
orientation  could  be  seen  in  lower  confidence  at  orientations  of  0°  {M  =  1 .78,  SD  =  0.48)  and  180°  {M  = 

1 .86,  SD  =  0.50)  as  compared  to  45°  (M  =  2.59,  SD  =  0.50),  90°  {M  =  2.78,  SD  =  0.35),  and  1 35°  (M  = 
2.62,  SD  =  0.48).  Post  hoc  testing  revealed  that  confidence  at  0°  and  1 80°  did  indeed  differ  significantly 
from  operator  confidence  at  the  oblique  and  broadside  orientations,  p  <  .0003  in  each  case.  No  other 
differences  were  statistically  significant. 

The  Resolution  x  Orientation  interaction  is  portrayed  in  Figure  34.  As  can  be  seen  in  the  figure, 
operator  confidence  was  nearly  identical  at  the  2  ft  and  4  ft  resolutions  when  vehicle  orientation  was  45°, 
90°,  or  135°.  However,  at  orientations  of  0°  and  180°,  operator  confidence  was  noticeably  higher  at  the  2 
ft  resolution.  In  fact,  follow-up  testing  indicated  that  the  two  levels  of  image  resolution  within  each 
orientation  differed  significantly  only  at  the  head-on  and  tail-on  orientations,  p  <  .006. 
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Figure  34:  Mean  operator  confidence  as  a  function  of  vehicle  orientation  at  the  2  ft  and  4  ft 
resolutions. 

The  interaction  between  clutter  and  number  of  confusers  is  portrayed  in  Figure  35,  which  shows 
that  confidence  did  not  vary  in  the  low,  medium,  and  high  clutter  backgrounds  when  zero  or  three 
confusers  were  present.  On  the  other  hand,  when  one  confuser  was  present,  confidence  appeared  to  be 
higher  in  the  low  and  medium  clutter  backgrounds  as  compared  to  high.  A  similar  situation  occurred 
when  two  confusers  were  present,  though  the  difference  was  smaller.  The  results  of  post-hoc  correlated 
/-tests  indicated  that  operator  confidence  was  indeed  significantly  higher  in  the  low  and  medium  clutter 
backgrounds  as  opposed  to  high  clutter  when  one  or  two  confusers  were  present,  p  <  .008  in  each  case. 
No  other  differences  were  statistically  significant. 


Figure  35:  Mean  operator  confidence  as  a  function  of  number  of  confusers  for  the  low  (L), 
medium  (M),  and  high  (H)  clutter  backgrounds. 


73 


The  Number  of  Confusers  x  Orientation  interaction  is  depicted  in  Figure  36.  The  plot  in  Figure 
36  shows  that  operator  confidence  was  highest  when  one  confuser  was  present  at  vehicle  orientations  of 
0°  and  1 80°.  However,  at  the  remaining  three  orientations,  confidence  was  lowest  when  one  confuser 
was  present.  Follow-up  testing  revealed  no  significant  differences  among  the  four  numbers  of  confusers 
at  orientations  of  45°  and  1 35°.  At  0°,  the  zero-one  (p  <  .0004),  zero-two  (p  <  .0052),  and  one-two  {p  < 
.0059)  differences  were  significant.  At  90°,  the  zero-one  {p  <  .0034),  one-two  {])  <  .0075),  and  one-three 
ip  <  .0045)  differences  were  significant.  And  at  1 80°,  the  zero-one  {p  <  .0005)  and  one-two  (/;  <  .0036) 
differences  were  significant. 


Figure  36:  Mean  operator  confidence  as  a  function  of  vehicle  orientation  for  each  number  of 
confusers. 

The  three-way  interaction  between  resolution,  number  of  confusers,  and  orientation  is  portrayed 
in  the  two  plots  of  Figure  37.  The  first  plot  shows  mean  operator  confidence  as  a  function  of  vehicle 
orientation  for  each  number  of  confusers  at  the  2  ft  resolution.  The  second  plot  shows  the  same 
relationship  at  the  4  ft  resolution.  In  general,  the  two  plots  show  that  confidence  tended  to  be  highest 
when  one  confuser  was  present  at  orientations  of  0°  and  1 80°.  At  the  remaining  three  orientations, 
'^confidence  was  lowest  when  one  confuser  was  present.  This  trend  appeared  at  both  the  2  ft  and  4  ft 
resolutions,  but  was  more  pronounced  at  4  ft.  The  results  of  post-hoc  correlated  r-tests  showed  that  at  the 
2  ft  resolution  and  an  orientation  of  0°,  the  zero-one  ip  <  .0086)  and  zero-two  {p  <  .0024)  differences 
were  significant;  at  180°,  only  the  one-two  difference  attained  significance  ip  <  .0018).  No  other 
differences  were  statistically  significant  at  the  2  ft  resolution.  At  the  4  ft  resolution,  the  zero-one,  one- 
two,  and  one-three  differences  were  significant  at  orientations  of  0°  and  90°  (p  <  .01  in  each  case).  At 
180°,  one  confuser  differed  significantly  from  zero  ip  <  .0001)  and  two  ip  <  .0097). 
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2  ft  Resolution 


4  ft  Resolution 


Figure  37:  Mean  operator  confidence  as  a  function  of  vehicle  orientation  for  each  number  of 
confusers  at  the  2  ft  (left)  and  4  ft  (right)  resolutions. 


Receiver  Operating  Characteristic  Curves 

To  portray  the  tradeoffs  between  hits  and  false  alarms  as  operator  response  bias  shifts  from 
conservative  to  lenient,  ROC  curves  were  generated  from  operators’  confidence  rating  data  in  the  present 
study.  In  doing  so,  the  original  six-point  rating  scale  was  retained  to  provide  five  operating  characteristic 
points  on  each  curve.  Because  these  are  continuous  curves  that  begin  at  hit/false  alarm  proportions  of 
(0,0)  and  end  at  (1,1),  they  are  particularly  useful  for  estimating  tradeoffs  beyond  what  actually  occurred 
in  the  empirical  investigation.  In  this  section,  ROC  curves  for  resolution,  clutter,  number  of  confusers, 
and  orientation  are  depicted.  Additional  curves  for  each  two-way,  three-way,  and  four-way  interaction 
can  be  found  in  the  appendices. 


Figure  38:  ROC  curves  for  resolution. 


Figure  39:  ROC  curves  for  background 
clutter. 
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Figure  40:  ROC  curves  for  number  of  Figure  41:  ROC  curves  for  vehicle 

confusers.  orientation. 

As  can  be  seen  by  examining  the  ROC  curves  for  each  main  effect,  overall  performance  was 
quite  good.  AI!  of  the  curves  were  located  in  the  upper  left  corner  of  the  ROC  space,  signifying  high  d 
scores  at  all  levels  of  each  independent  variable.  In  particular,  the  ROC  curves  for  resolution  show  that 
the  2  ft  and  4  ft  resolutions  were  virtually  indistinguishable  from  each  other  (Figure  38).  A  similar 
situation  can  be  seen  in  the  ROC  curves  for  number  of  confusers  (Figure  40).  With  respect  to 
background  clutter,  on  the  other  hand,  there  was  a  slight  difference  between  the  curves  for  low  and  high 
clutter  (Figure  39).  The  curve  for  low  clutter  was  positioned  slightly  higher  in  the  upper  left  corner  than 
the  curve  for  high  clutter,  implying  that  operator  sensitivity  was  somewhat  greater  in  the  low  clutter 
scenes.  Finally,  the  ROC  curves  for  orientation  in  Figure  41  show  that  the  curves  for  45^  90°,  and  135° 
were  located  above  the  curves  representing  0°  and  180°.  This  relationship  signifies  that  the  task  of  target 
detection  was  relatively  easier  when  the  vehicle  was  positioned  broadside  or  obliquely  as  opposed  to 
either  head-on  or  tail-on.  It  should  be  emphasized,  however,  that  even  the  differences  associated  with 
background  clutter  and  vehicle  orientation  were  exceedingly  small.  For  example,  when  the  orientation 
was  0°,  a  false  alarm  rate  of  5%  was  accompanied  by  a  hit  rate  of  about  95%.  When  the  orientation  was 
90°,  the  same  false  alarm  rate  was  accompanied  by  a  higher  hit  rate  of  99%. 

Summary  of  Results 

In  summary,  the  effects  of  resolution,  clutter,  number  of  confusers,  and  vehicle  orientation  on 
operators’  hits,  false  alarms,  sensitivity,  bias,  RT  for  hits,  RT  for  correct  rejections,  and  decision-making 
confidence  were  assessed  by  means  of  four-way  repeated-measures  ANOVAs  and  post-hoc  correlated  t- 
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tests.  The  outcomes  of  these  statistical  analyses  indicated  first  that  image  resolution  affected  decision¬ 
making  time  and  confidence  only  and  not  performance  accuracy  per  se.  Specifically,  the  RT  for  both  hits 
and  correct  rejections  was  elevated  and  confidence  was  poorer  at  the  4  ft  resolution  as  compared  to  2  ft. 
However,  there  were  no  statistically  significant  differences  in  hits,  false  alarms,  sensitivity,  or  response 
bias  between  the  two  resolutions.  Second,  the  effects  of  background  clutter  could  be  seen  in  operators’ 
sensitivity  to  TEL  target  detection  and  localization,  their  RT  for  both  hits  and  correct  rejections,  and  their 
decision-making  confidence.  Sensitivity  was  significantly  poorer  in  the  high  clutter  background  than  in 
low  clutter,  and  RT  was  slower  and  confidence  was  lower  in  high  clutter  as  compared  to  low  and 
medium.  Third,  with  respect  to  the  independent  variable  of  primary  interest  in  this  study,  the  number  of 
confusers  did  not  impact  operator  performance  accuracy  or  confidence;  however,  the  RT  for  hits  and 
correct  rejections  was  affected.  Namely,  for  both  dependent  variables,  the  RT  was  slower  when  two  or 
three  confusers  were  present  as  opposed  to  when  only  zero  or  one  appeared  in  the  scene.  Fourth,  vehicle 
orientation  affected  perceptual  sensitivity,  RT  for  hits,  RT  for  correct  rejections,  and  confidence.  In 
general,  performance  accuracy,  reaction  time,  and  confidence  tended  to  be  poorer  at  orientations  of  O'" 
and  180°;  sensitivity  was  significantly  reduced  at  0°  and  180°  as  compared  to  45°  and  135°;  the  RT  for 
hits  was  slower  at  0°  and  180°  as  compared  to  45°,  90°,  and  135°;  the  RT  for  correct  rejections  was  worse 
at  0°  as  compared  to  45°,  90°,  and  1 35°;  and  confidence  was  significantly  lower  at  orientations  of  0°  and 
1 80°  as  compared  to  45°,  90°,  and  1 35°. 

Finally,  three  of  the  eleven  interactions  are  worth  mentioning.  The  Clutter  x  Orientation 
interaction  was  statistically  significant  in  all  analyses  except  the  RT  for  correct  rejections  and  operator 
confidence.  The  interactions  stemmed  from  a  tendency  for  performance  to  deteriorate  in  the  high  clutter 
background  at  vehicle  orientations  of  0°  and  180°.  For  example,  false  alarms  were  higher  and  operator 
sensitivity  was  significantly  lower  in  high  clutter  as  compared  to  low  and  medium  at  a  vehicle  orientation 
of  1 80°.  At  an  orientation  of  0°,  the  proportion  of  hits  was  lower  in  high  clutter  as  opposed  to  low  and 
the  RT  for  hits  increased  progressively  as  the  background  clutter  increased. 

The  Resolution  x  Number  of  Confusers  x  Orientation  interaction  was  statistically  significant  in 
all  analyses  except  response  bias.  For  hits,  false  alarms,  and  sensitivity,  the  interactions  resulted  from 
differences  among  the  number  of  confusers  at  the  2  ft  resolution  when  the  vehicle  orientation  was  180°. 
For  instance,  in  this  condition,  false  alarms  were  higher  when  two  confusers  were  present  as  compared  to 
zero,  one,  and  three;  and  sensitivity  was  greater  for  zero  confusers  as  compared  to  two.  Both  hits  and 
sensitivity  were  also  inexplicably  higher  when  three  confusers  were  present  as  compared  to  two.  With 
respect  to  RT  for  hits  and  correct  rejections,  the  interactions  involved  differences  among  the  number  of 


77 


confusers  within  each  orientation  at  both  the  2  ft  and  4  ft  resolutions.  When  significant  differences 
occurred,  they  always  revealed  a  slower  reaction  time  for  the  higher  number  of  confusers.  The  three-way 
interaction  was  somewhat  unique  with  regard  to  operator  confidence.  In  general,  for  both  the  2  ft  and  4  ft 
resolutions,  operator  confidence  tended  to  be  highest  when  one  confuser  was  present  at  orientations  of  0° 
and  180°. 

And,  last,  the  Clutter  x  Number  of  Confusers  x  Orientation  interaction  was  significant  in  all  of 
the  analyses  except  confidence.  For  hits,  false  alarms,  sensitivity,  and  response  bias,  the  interactions 
occurred  as  a  result  of  performance  in  the  high  clutter  background  at  a  vehicle  orientation  of  180°.  In  this 
condition,  performance  tended  to  be  worst  when  two  confusers  were  pre.sent:  both  hits  and  sensitivity 
were  lower  for  two  confusers  as  compared  to  one  and  three;  false  alarms  were  higher  for  two  confusers 
as  compared  to  zero  and  three;  and  bias  was  more  conservative  for  two  confusers  as  opposed  to  three. 

For  the  RT  variables,  on  the  other  hand,  the  interactions  stemmed  from  variations  among  the  number  of 
confusers  within  each  orientation  in  all  of  the  backgrounds.  Again,  when  significant  differences 
occurred,  they  always  indicated  that  reaction  time  was  slower  for  the  higher  number  of  confusers. 

DISCUSSION 

As  described  in  the  Introduction,  the  purpose  of  the  present  study  was  to  quantify  the  effects  of 
the  presence  of  confu.sers  on  TEL  target  detection  and  localization  performance  with  SAR  imagery, 
primarily  in  terms  of  the  signal  detection  theory  measures  of  sensitivity  and  bias.  A  secondary  goal  was 
to  examine  the  impact  of  TEL  orientation  on  performance  effectiveness.  Finally,  the  effects  of 
background  clutter  and  image  resolution  were  also  studied.  We  expected  that  performance  effectiveness 
would  decline  1)  as  the  number  of  confusers  in  the  scene  increased,  2)  as  the  orientation  deviated  from  a 
90°  broadside  view  of  the  TEL,  3)  as  the  background  clutter  increased,  and  4)  as  the  resolution 
deteriorated  from  2  ft  to  4  ft. 

Effects  of  Confusers  on  TEL  Target  Detection  and  Localization  Performance. 

With  respect  to  the  primary  independent  variable  of  interest,  the  results  indicated  that  the  number 
of  confusers  did  not  impact  operator  hits,  false  alarms,  sensitivity,  bias,  or  decision-making  confidence. 
The  effects  of  number  of  confusers  could  be  seen  only  in  operators’  reaction  times  to  the  images. 
Specifically,  both  RT  for  hits  and  RT  for  correct  rejections  were  slower  when  two  or  three  confusers 
were  present  in  a  scene  as  opposed  to  when  only  zero  or  one  appeared.  This  outcome  in  and  of  itself 
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does  not  necessarily  imply  a  detrimental  effect  of  confusers  on  performance  since  reaction  time  naturally 
increases  as  the  number  of  objects  to  be  inspected  increases,  regardless  of  whether  or  not  they  are 
confusers.  Hence,  the  slower  reaction  times  when  two  or  more  confusers  were  present  may  be 
attributable  solely  to  the  need  to  examine  more  objects  before  reaching  a  decision  as  opposed  to  requiring 
additional  time  to  decide  which  object  was  the  TEL  target.  Along  these  lines,  it  is  noteworthy  that 
decision-making  time  did  not  differ  between  target-only  scenes  (zero  confusers)  and  target-plus-one- 
confuser  scenes.  When  only  a  single  confuser  was  present  in  a  target  scene,  operators  were  still  able  to 
determine  quickly  which  object  was  the  TEL  (i.e.,  the  presence  of  the  confuser  did  not  significantly 
increase  total  reaction  time  and  operators  apparently  did  not  need  more  time  to  reach  a  target/nontarget 
decision  when  the  confuser  was  present).  It  was  only  when  two  or  more  confusers  were  present  that  RT 
increased,  most  likely  as  a  consequence  of  the  simple  need  to  examine  more  objects.  Thus,  our 
hypothesis  that  performance  effectiveness  would  decline  as  the  number  of  confusers  increased  was  only 
partially  supported  by  the  results. 

Recall  that  previous  literature  has  shown  that  the  time  to  acquire  a  target  increases  and 
performance  accuracy  deteriorates  as  the  number  of  confusers  increases  (Greening,  1974).  The  results  of 
the  present  study  concur  in  terms  of  the  increase  in  reaction  time  but  not  with  regard  to  performance 
accuracy.  The  reasons  for  this  apparent  discrepancy  may  lie  in  the  very  definition  of  what  is  and  what  is 
not  a  confuser.  First,  according  to  Greening  (1974),  a  confuser  can  be  thought  of  as  an  object  that 
resembles  the  target  to  a  greater  or  lesser  degree.  He  further  indicates  that  the  distinction  between  clutter 
and  confusers  is  not  always  clear-cut  in  the  real  world.  In  general,  he  regards  clutter  as  more  or  less 
homogeneous  aspects  of  the  background  that  can  readily  be  dismissed  as  nontargets,  A  confuser  is  any 
part  of  the  background  that  resembles  the  target  closely  enough  to  warrant  further  examination.  With  this 
in  mind,  the  M-548  cargo  truck  can  indeed  be  considered  a  confuser  rather  than  clutter  because  it  did 
resemble  the  target  to  some  degree,  though  this  degree  was  lesser  rather  than  greater.  As  noted  in  the 
Results,  when  operators  believed  a  target  to  be  present  in  the  image,  they  correctly  selected  the  TEL 
99.3%  of  the  time  and  incorrectly  chose  the  confuser  only  0.5%  of  the  time.  Hence,  it  would  seem  that 
the  absence  of  an  effect  of  number  of  confusers  on  performance  accuracy  was  due  in  part  to  only  a  very 
slight  resemblance  of  the  confuser  to  the  target.  That  is,  while  the  M-548  was  a  confuser  vehicle,  it  was 
not  particularly  confusing. 

Further  evidence  along  these  lines  comes  from  a  more  structured  analysis  of  the  similarity 
between  the  TEL  and  the  M-548  cargo  truck  akin  to  the  type  conducted  by  O’ Kane,  Biederman,  Cooper, 
and  Nystrom  (1997).  As  discussed  earlier,  their  investigations  indicated  that  a  similarity  metric  based  on 


79 


the  magnitude  and  number  of  viewpoint-invariant  part  differences  (e.g.,  wheeled  versus  tracked;  presence 
or  absence  of  a  cargo  bed;  presence  or  absence  of  a  gun)  between  a  pair  of  vehicles  was  significantly 
related  to  performance  effectiveness.  If  we  examine  the  number  of  viewpoint-invariant  part  differences 
between  the  TEL  and  M-548,  we  see  that  the  two  vehicles  were  actually  not  very  similar  at  all.  First  and 
foremost,  the  TEL  is  wheeled  whereas  the  M-548  is  tracked.  This  alone  would  place  the  vehicles  on 
opposite  sides  of  the  trunk  in  O’Kane  et  al.’s  (1997)  similarity  tree  (making  them  more  dissimilar  than 
similar).  Further,  the  TEL  has  a  missile  on  the  back  of  the  vehicle  and  the  M-548  has  a  gun  at  the  front. 
Finally,  the  M-548  has  a  cargo  bed  and  the  TEL  does  not.  Thus,  according  to  O’ Kane  et  al.’s  analysis, 
these  are  two  vehicles  that  are  not  very  likely  to  be  confused  because  they  are  rather  far  apart  on  the 
similarity  tree. 

Of  course,  the  efficacy  of  this  similarity  metric  is  based  on  the  premise  that  the  distinguishing 
characteristics  between  the  two  vehicles  are  prominent  in  the  imagery.  It  may  be  the  case  that  the 
imagery  is  so  poor  (e.g.,  the  4  ft  resolution)  that  only  size  and  shape  are  apparent.  If  this  happens, 
operators  would  have  to  depend  not  on  the  viewpoint-invariant  parts  but  on  the  less  reliable  metric 
properties  of  the  vehicles.  In  this  regard  even,  the  two  vehicles  were  more  dissimilar  than  similar;  the 
TEL  is  approximately  13  m  long  whereas  the  M-548  is  only  about  6  m  long.  We  should  note  that  the  M- 
548  was  selected  as  the  confuser  vehicle  because  it  was  explicitly  designated  as  a  TEL  confuser  in  the 
MSTAR  database  and  is  also  frequently  regarded  as  such  in  the  ATR  community.  However,  as  was 
evidenced  in  the  present  study,  the  TEL  was  actually  easily  distinguishable  from  the  M-548  confuser 
vehicle  (except  possibly  at  tail-on  or  head-on  orientations). 

Effects  of  Vehicle  Orientation  on  TEL  Target  Detection  and  Localization  Performance. 

In  fact,  the  results  did  indicate  that  TEL  target  detection  and  localization  performance  was  most 
problematic  at  orientations  of  0°  (head-on)  and  180°  (tail-on)  as  opposed  to  45°,  90°,  and  135°.  Operator 
sensitivity  was  significantly  reduced  at  0°  and  180°  as  compared  to  45°  and  135°;  the  RT  for  hits  was 
slower  at  0°  and  1 80°  as  compared  to  45°,  90°,  and  1 35°;  the  RT  for  correct  rejections  was  worse  at  0°  as 
compared  to  45°,  90°,  and  1 35°;  and  confidence  was  significantly  lower  at  orientations  of  0°  and  1 80°  as 
compared  to  45°,  90°,  and  135°.  Further,  in  the  significant  interactions  involving  vehicle  orientation,  0° 
and  180°  invariably  emerged  as  the  orientations  associated  with  the  worst  performance.  The  relative 
difficulty  of  the  task  at  the  head-on  and  tail-on  orientations  most  likely  resulted  from  the  fact  that  the 
features  distinguishing  the  TEL  from  the  M-548  were  more  difficult  to  discern  at  0°  and  1 80°.  In  the 
broadside  and  oblique  views  of  the  TEL,  the  size  difference  was  more  readily  apparent  as  was  the 


80 


difference  in  the  radar  returns  of  the  two  vehicles.  Specifically,  the  machine  gun  mounted  on  the  top  of 
the  M-548’s  cab  produced  a  noticeable  “tail”  in  its  return,  which  was  completely  different  from  the 
TEL’s  radar  return.  This  “tail”  was  least  visible  when  the  M-548  was  oriented  at  0°  or  180°,  which  made 
it  more  difficult  to  distinguish  between  the  TEL  and  the  confuser  at  those  orientations  and  thereby 
produced  somewhat  poorer  performance. 

These  outcomes  are  noteworthy  because  they  provide  the  first  empirical  evidence  of  the  effects 
of  orientation  on  TEL  target  detection  and  localization  performance.  As  described  in  the  Introduction, 
studies  of  orientation  to  date  have  focused  on  ship  recognition  rather  than  ground  targets.  While  the 
outcomes  of  ship  recognition  studies  may  generalize  to  some  extent  to  studies  of  target  acquisition  with 
land  vehicles,  the  results  of  the  present  study  indicate  that  they  may  not  wholly  apply.  For  example,  the 
maritime  studies  demonstrated  a  clear  advantage  for  a  90°  ship  orientation  over  orientations  ranging  from 
10°  off  bow  up  to  90°  (Mocharnuk,  Gaudio,  &  Suwe,  1981;  Whitehurst,  1976).  Although  the  broadside 
view  of  the  TEL  was  superior  to  head-on  and  tail-on  views  in  the  present  investigation,  it  was  not 
superior  to  the  oblique  orientations  of  45°  and  1 35°.  In  general,  performance  effectiveness  did  not  differ 
significantly  among  orientations  of  45°,  90°,  and  135°.  The  difference  in  outcomes  between  the  present 
study  and  those  of  the  ship  recognition  studies  may  also  have  been  due  in  part  to  the  different  types  of 
imagery  used  in  each  investigation.  Mocharnuk,  Gaudio,  and  Suwe  (1981)  used  simulated  IR  imagery 
and  Whitehurst  (1976)  used  television  imagery,  whereas  we  used  simulated  SAR  imagery.  Due  to 
geometry  and  viewing  angle,  there  was  relatively  less  information  in  the  IR  and  television  imagery  as  the 
targets  deviated  from  a  broadside  view  than  would  be  true  for  SAR  imagery.  Because  SAR  imagery 
provides  a  “plan  view”  (Toomay,  1998),  it  always  displayed  the  imaged  area  as  if  viewed  from  directly 
overhead.  As  vehicle  orientation  changed,  the  SAR  imagery  changed  due  to  radar  reflecting  off  different 
surfaces;  hence,  what  could  not  be  seen  due  to  occlusion  with  IR  or  television  imagery  was  marked  with 
a  shadow  in  radar.  In  sum,  both  differences  in  the  type  of  imagery  (i.e.,  SAR,  IR,  television)  and  the  type 
of  vehicles  (i.e.,  land  versus  sea)  used  may  have  contributed  to  the  discrepancies  in  outcomes  between 
our  study  and  previous  examinations  of  vehicle  orientation. 

Effects  of  Resolution  and  Background  Clutter  on  TEL  Target  Detection  and  Localization 

Performance. 

Finally,  in  addition  to  the  effects  of  number  of  confusers  and  vehicle  orientation,  the  impact  of 
image  resolution  and  background  clutter  on  TEL  target  acquisition  was  also  examined  in  the  current 
study.  Image  resolution  affected  operators’  reaction  times  to  the  imagery  and  their  decision-making 
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confidence  only  and  not  their  ability  to  detect  and  locate  the  TEL  itself.  Operators  took  slightly  longer  to 
reach  a  decision  at  the  4  ft  image  resolution  as  compared  to  2  ft,  and  they  had  less  confidence  in  that 
decision.  While  this  outcome  is  not  unexpected,  it  only  partially  supports  our  hypothesis  since  we  also 
expected  to  see  some  variations  in  hits,  false  alarms,  or  sensitivity.  As  described  earlier,  previous  studies 
have  demonstrated  that  performance  accuracy  typically  improves  along  with  improvements  in  the 
resolution  (Davis,  See,  Shacklett,  &  Kuperman,  1996;  Kuperman,  Wilson,  &  Davis,  1993;  Kuperman, 
Wilson,  &  Perez,  1988;  See  &  Kuperman,  1995).  In  the  Davis  et  al.  (1996)  study,  for  example, 
sensitivity  was  significantly  higher  and  false  alarms  were  lower  at  a  resolution  of  4  ft  as  compared  to  8.5 
ft.  Comparable  outcomes  may  not  have  emerged  in  the  present  study  due  to  the  exceptionally  high  level 
of  performance  accuracy.  Overall,  the  mean  proportion  of  hits  was  .93  and  the  mean  proportion  of  false 
alarms  was  only  .04,  indicating  that  operators  were  very  accurate  and  made  few  mistakes.  Further,  the 
mean  level  of  sensitivity,  d  =  3.4,  indicated  that  the  task  lay  within  a  moderately  ea.sy  range  rather  than 
difficult. 

With  respect  to  background  clutter,  operator  sensitivity  was  significantly  poorer  in  the  high 
clutter  background  as  opposed  to  low  clutter.  In  addition,  both  RT  for  hits  and  RT  for  correct  rejections 
were  slower  and  confidence  was  poorer  in  high  clutter  as  compared  to  medium  and  low.  These  findings 
concur  with  the  results  of  previous  investigations,  which  have  demonstrated  that  performance  typically 
deteriorates  as  the  amount  of  background  clutter  in  a  scene  increases  (Cole  &  Hughes,  1984;  Davis,  See, 
Shacklett,  &  Kuperman,  1996;  Kuperman,  Wilson,  &  Perez,  1988;  See,  Davis,  &  Kuperman,  1997;  See 
&  Kuperman,  1995). 

We  should  note  here  that  clutter  was  defined  as  the  number  of  trees  per  image  in  the  present 
study  rather  than  the  number  of  trees  per  square  mile,  as  in  previous  studies  (e.g..  See  &  Kuperman, 

1995).  Thus,  increases  in  background  clutter  were  represented  as  increments  in  the  number  of  trees  from 
15  (low)  to  45  (medium)  to  135  (high).  Hence,  as  the  number  of  trees  increased,  the  image  appeared  to 
be  more  cluttered.  However,  when  image  resolution  is  simultaneously  taken  into  account,  the  low, 
medium,  and  high  levels  of  clutter  may  not  have  been  comparable  between  the  2  ft  and  4  ft  resolutions, 
precisely  because  clutter  was  defined  as  the  number  of  trees  per  image.  Since  image  size  remained 
constant,  the  image  covered  a  much  larger  area  as  the  resolution  worsened  from  2  ft  to  4  ft.  Thus,  the 
same  number  of  trees  appeared  to  occupy  much  less  space,  creating  more  open  expanses  in  the  image. 
This  effect  was  particularly  visible  at  the  highest  clutter  level.  At  the  2  ft  resolution,  135  trees  occupied 
most  of  the  image,  making  it  appear  extremely  cluttered.  At  the  4  ft  resolution,  however,  the  amount  of 
open  space  was  relatively  greater,  potentially  making  it  easier  to  detect  any  vehicles  that  might  be 
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present.  In  fact,  there  was  a  tendency  for  hits  and  sensitivity  to  be  higher  at  the  4  ft  resolution  when 
clutter  was  high,  though  this  difference  was  not  significant.  In  future  studies  that  simultaneously 
examine  both  resolution  and  clutter,  background  clutter  should  best  be  defined  in  terms  of  the  area 
covered. 

Future  Research, 

First,  future  research  should  seek  to  examine  further  the  effects  of  confusers  on  TEL  target 
acquisition  and  augment  the  present  findings  by  using  vehicles  that  are  potentially  more  confusing  than 
the  M-548.  One  candidate  is  the  M-520,  a  cargo  truck  whose  size  (9.8  m  long)  more  closely  emulates 
that  of  the  TEL  (13m  long)  than  is  true  for  the  M-548  (6  m  long).  In  addition,  the  M-520,  like  the  TEL, 
is  a  wheeled  vehicle.  While  the  present  study  was  being  completed,  a  CAD  model  of  the  M-520  became 
available  through  the  MSTAR  database,  enabling  it  to  be  used  in  future  examinations  of  TEL  confusers. 

Second,  future  work  should  test  the  ideas  of  Duncan  and  Humphreys  (1989)  for  applicability  to 
real-world  tasks  such  as  TEL  target  acquisition  in  the  presence  of  more  than  one  type  of  confuser.  As 
described  in  the  Introduction,  Duncan  and  Humphreys  demonstrated  that  the  effects  of  both  T/NT 
similarity  and  NT/NT  similarity  must  be  taken  into  account  when  assessing  the  effects  of  confusers  on 
target  detection  performance.  Specifically,  they  showed  that  search  difficulty  increased  as  T/NT 
similarity  increased  and  as  NT/NT  similarity  decreased.  Further,  the  most  rapid  search  times  occurred 
when  the  target  and  nontargets  were  dissimilar  and  all  of  the  nontargets  were  similar.  Because  only  one 
type  of  confuser  was  used  in  the  current  study,  the  direct  applicability  of  these  outcomes  could  not  be 
evaluated  with  regard  to  TEL  target  acquisition.  The  effects  of  confuser  orientation  similarity  could 
have  been  explored  had  we  had  sufficient  imagery.  However,  our  primary  interest  was  in  the  effects  of 
TEL  orientation  on  target  acquisition  performance.  The  orientations  of  the  confusers,  when  present, 
varied  randomly;  hence,  in  some  images,  two  or  more  confusers  had  the  same  orientation.  In  such  cases, 
the  spreading  suppression  proposed  by  Duncan  and  Humphreys  should  presumably  be  exceptionally 
strong:  the  nontargets  not  only  represent  the  same  type  of  vehicle  but  also  have  radar  returns  that  are 
identical  in  every  way.  Unfortunately,  there  was  not  enough  imagery  to  study  this  effect  in  isolation. 
Further  examination  of  the  effects  of  NT/NT  similarity  in  terms  of  both  confuser  type  and  confuser 
orientation  could  prove  to  be  useful,  particularly  since  it  is  not  known  whether  Duncan  and  Humphreys’ 
(1989)  results  generalize  to  real-world  tasks. 

Finally,  future  work  should  also  examine  the  effects  of  TEL  confusers  on  eye  gaze  behavior,  as 
in  studies  conducted  by  Boersema,  Zwaga,  and  Adams  (1989)  and  Boersema  and  Zwaga  (1993).  Their 
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investigations  showed  that  observers’  scanning  patterns  were  less  efficient  when  confusers  were  present. 
Observers  took  longer  to  locate  a  target  sign  and  made  more  eye  fixations  as  the  number  of  confuser 
advertisements  in  the  scene  increased.  However,  not  all  of  the  extra  eye  fixations  actually  fell  on  the 
confusers,  implying  that  the  extra  time  was  spent  not  just  in  looking  at  the  additional  confusers  but  in 
looking  less  efficiently  at  the  scene  in  general.  Future  endeavors  should  seek  to  determine  if  these 
notions  apply  to  the  acquisition  of  TEL  targets  when  confusers  are  present.  Sub.sequent  work  might  then 
focus  on  the  development  of  training  methods  to  improve  scanning  patterns. 

CONCLUSIONS  . 

1 .  In  terms  of  hits,  false  alarms,  perceptual  sensitivity,  response  bias,  and  operator  confidence,  TEL 
target  acquisition  performance  was  unaffected  as  the  number  of  M-548  confusers  increased  from  zero 
to  four. 

2.  Reaction  time  was  somewhat  slower  (approximately  300  milliseconds)  when  two  or  more  M-548 
confusers  were  present  as  compared  to  when  only  zero  or  one  confusers  appeared  in  the  scene. 

3.  TEL  target  acquisition  performance  was  superior  and  confidence  was  higher  at  vehicle  orientations 
of  45°,  90°,  and  135°  as  opposed  to  0°  (head-on)  and  180°  (tail-on). 

4.  Decision-making  time  was  slower  and  confidence  was  poorer  at  an  image  resolution  of  4  ft  as 
compared  to  2  ft. 

5.  Perceptual  .sensitivity,  reaction  time,  and  confidence  were  all  degraded  when  the  background  clutter 
was  high. 

6.  Future  work  should  examine  the  effects  of  confusers  on  TEL  target  acquisition  by  using: 

a)  a  confuser  such  as  the  M-520  that  more  closely  resembles  the  TEL 

b)  more  than  one  type  of  confuser  vehicle  to  examine  the  effects  of  T/NT  and  NT/NT  similarity  on 
TEL  target  acquisition 

c)  equipment  to  assess  eye  gaze  behavior  when  confusers  are  present 
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APPENDIX  A 

ROC  CURVES  FOR  TWO-WAY  INTERACTIONS 
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Figure  A  2:  ROC  curves  for  Resolution  x  Number  of  Confusers. 
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Figure  A  3:  ROC  curves  for  Resolution  x  Orientation. 
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ROC  CURVES  FOR  THREE-WAY  INTERACTIONS 


94 


P(H)  p{H)  p(H)  p(H) 


1.0  jop 
0.8  ' 
0.6 

0.4  > 

0.2  ► 

0.0  A- 


2  ft  Resolution 
0  Confuse  rs 

— e — Low 
■— B — Modium 
— A— Hgh 


0.4  0.6  0.8  1.0 

P(FA) 


1.0 

0.8 

0.6 

0.4 

0.2 

0.0 


4  ft  Resolution 
0  Confuse  rs 


Low 

Mooiuin 

High 

0.0  0.2  0.4  0.6  0.8  1.0 

p(FA) 


1.0 


0.6 

0.4 


2  ft  Resolution 
1  Confuser 


-  Low 

-Medium 

-High 


0.2 

0.0 


0.0  0.2  0.4  0.6  0.8  1.0 

P(FA) 


p(FA) 


2  ft  Resolution 

2  Confusers  j 

■  O  Low 

—B—*  Medium 

—A— High 

0.0  0.2  0.4  0.6  0.8  1.0 

P(FA) 


1.0 

0.8 

0.6  j 

. 

X 

a 

0.4 

'  2  ft  Resolution 

- . 3  Confusers  - 

o  Low 

0.2 

— B—  Medium 

—A —  High 

0.0  1 

-  -  A  I  t  1  1 

0.0  0.2  0.4  0.6  0.8  1.0 

p(FA) 


0.6  ■  -  - 

X 

a  i  I 

0.4  •  -  -  - 

0.2  ►  -  -  - 

0.0  - ^ 

0.0  0.2 


4  ft  Resolution 
3  Confuse  rs 


Low 

Medium 

High 

0.4  0.6  0.8  1.0 

p(FA) 


Figure  B  1;  ROC  curves  for  Resolution  x  Clutter  x  Number  of  Confusers. 
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Figure  B  2:  ROC  Curves  for  Resolution  x  Clutter  x  Orientation, 
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Figure  B  3;  ROC  curves  for  Resolution  x  Number  of  Confusers  x  Orientation. 
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Figure  B  4:  ROC  curves  for  Clutter  x  Number  of  Confusers  x  Orientation,  {continued  in  Figure  B5) 
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Figure  B  5:  ROC  curves  for  Clutter  x  Number  of  Confusers  x  Orientation,  {continued  from  Figure 

B4). 
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APPENDIX  C 

ROC  CURVES  FOR  FOUR-WAY  INTERACTIONS 


100 


P(H)  P(H) 


p(FA)  p{FA) 


Figure  C  1:  ROC  curves  for  0, 1, 2,  and  3  confusers  at  the  2  ft  resolution,  low  clutter,  0^ 


101 


P(H)  p(H) 


P(FA)  p(FA) 


Figure  C  2:  ROC  curves  for  0, 1,  2,  and  3  confusers  at  the  2  ft  resolution,  low  clutter,  45". 
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Figure  C  3:  ROC  curves  for  0, 1, 2,  and  3  confusers  at  the  2  ft  resolution,  low  clutter,  90”. 
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Figure  C  4:  ROC  curves  for  0, 1, 2,  and  3  confusers  at  the  2  ft  resolution,  low  clutter,  135". 
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Figure  C  5:  ROC  curves  for  0, 1, 2,  and  3  confusers  at  the  2  ft  resolution,  low  clutter,  180”. 
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Figure  C  6:  ROC  curves  for  0, 1,  2,  and  3  confusers  at  the  2  ft  resolution,  medium  clutter,  0". 


106 


P(H)  p(H) 


p(FA)  P(FA) 


Figure  C  7:  ROC  curves  for  0, 1,  2,  and  3  confusers  at  the  2  ft  resolution,  medium  clutter,  45“. 
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Figure  C  8:  ROC  curves  for  0, 1,  2,  and  3  confusers  at  the  2  ft  resolution,  medium  clutter,  90". 
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Figure  C  9:  ROC  curves  for  0, 1,  2,  and  3  confusers  at  the  2  ft  resolution,  medium  clutter,  135". 
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Figure  C  10:  ROC  curves  for  0, 1, 2,  and  3  confusers  at  the  2  ft  resolution,  medium  clutter,  180". 
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Figure  C  11:  ROC  curves  for  0, 1,  2,  and  3  confusers  at  the  2  ft  resolution,  high  clutter,  0". 
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Figure  C  12:  ROC  curves  for  0, 1,  2,  and  3  confusers  at  the  2  ft  resolution,  high  clutter,  45". 
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Figure  C  13:  ROC  curves  for  0, 1, 2,  and  3  confusers  at  the  2  ft  resolution,  high  clutter,  90". 
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Figure  C  14:  ROC  curves  for  0, 1, 2,  and  3  confusers  at  the  2  ft  resolution,  high  clutter,  135". 
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Figure  C  15:  ROC  curves  for  0, 1, 2,  and  3  confusers  at  the  2  ft  resolution,  high  clutter,  180”. 
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Figure  C  16:  ROC  curves  for  0, 1,  2,  and  3  confusers  at  the  4  ft  resolution,  low  clutter,  0". 
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Figure  C  17:  ROC  curves  for  0, 1, 2,  and  3  confusers  at  the  4  ft  resolution,  low  clutter,  45”. 
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Figure  C  18:  ROC  curves  for  0, 1, 2,  and  3  confusers  at  the  4  ft  resolution,  low  clutter,  90“. 
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Figure  C  19:  ROC  curves  for  0, 1, 2,  and  3  confusers  at  the  4  ft  resolution,  low  clutter,  135". 
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Figure  C  20:  ROC  curves  for  0, 1, 2,  and  3  confusers  at  the  4  ft  resolution,  low  clutter,  180“. 
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Figure  C  21:  ROC  curves  for  0, 1, 2,  and  3  confusers  at  the  4  ft  resolution,  medium  clutter,  0®. 
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Figure  C  22:  ROC  curves  for  0, 1,  2,  and  3  confusers  at  the  4  ft  resolution,  medium  clutter,  45“. 
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Figure  C  23:  ROC  curves  for  0, 1, 2,  and  3  confusers  at  the  4  ft  resolution,  medium  clutter,  90". 
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Figure  C  24:  ROC  curves  for  0, 1, 2,  and  3  confusers  at  the  4  ft  resolution,  medium  clutter,  135“. 
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Figure  C  25:  ROC  curves  for  0, 1, 2,  and  3  confusers  at  the  4  ft  resolution,  medium  clutter,  180“. 
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Figure  C  26:  ROC  curves  for  0, 1,  2,  and  3  confusers  at  the  4  ft  resolution,  high  clutter,  0**. 
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Figure  C  27:  ROC  curves  for  0, 1, 2,  and  3  confusers  at  the  4  ft  resolution,  high  clutter,  45“. 
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Figure  C  28:  ROC  curves  for  0, 1,  2,  and  3  confusers  at  the  4  ft  resolution,  high  clutter,  90". 
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Figure  C  29:  ROC  curves  for  0, 1, 2,  and  3  confusers  at  the  4  ft  resolution,  high  clutter,  135". 
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Figure  C  30:  ROC  curves  for  0, 1,  2,  and  3  confusers  at  the  4  ft  resolution,  high  clutter,  180". 
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AFRL/SNAT 

Target  Recognition  Branch,  Sensor  and  ATR  Technology  Division,  Sensor  Directorate, 
of  the  Air  Force  Research  Laboratory 

ANOVA 

Analysis  of  Variance 

ATC/ATR 

Automatic  Target  Cueing/Recognition 

c 

Signal  Detection  Theory  Measure  of  Response  Bias 

CAD 

Computer-aided  Design 

CIWAL 

Crew  Aiding  and  Information  Warfare  Analysis  Laboratory 

d' 

Signal  Detection  Theory  Measure  of  Perceptual  Sensitivity 

FA 

False  Alarm 

H 

Hit 

IRN 

Instructor  Radar  Navigator 

IWSO 

Instructor  Weapons  System  Operator 

J-STARS 

Joint  Surveillance  Target  Attack  Radar  System 

LROC 

Localization  Receiver  Operating  Characteristic 

M 

Mean 

MBVLAB 

Model-based  Vision  Laboratory 

MSTAR 

Moving  and  Stationary  Target  Recognition 

N 

Noise 

NT 

Nontarget 

ROC 

Receiver  Operating  Characteristic 

RT 

Reaction  Time 

SAR 

Synthetic  Aperture  Radar 

SD 

Standard  Deviation 

SN 

Signal-plus-Noise 

T 

Target 

TAD 

Theater  Air  Defense 

TEL 

Transporter-Erector-Launcher 

TM 

Theater  Missile 

TSD 

Theory  of  Signal  Detection 

UAV 

Unmanned  Aerial  Vehicle 

VSTM 

Visual  Short  Term  Memory 

WSO 

Weapons  System  Operator 
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